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CENTRAL SWATHS 

(A GENERALIZATION OF THE CENTRAL PATH) 

JAMES RENEGAR 



Abstract. We develop a natural generalization to the notion of the central 
path — a notion that lies at the heart of interior-point methods for convex 
optimization. The generalization is accomplished via the "derivative cones" 
of a "hyperbolicity cone," the derivatives being direct and mathematically- 
5-H ' appealing relaxations of the underlying (hyperbolic) conic constraint, be it the 

^ . non-negative orthant, the cone of positive semidefinite matrices, or other. 

We prove that a dynamics inherent to the derivative cones generates paths 
always leading to optimality, the central path arising from a special case in 
OO ' which the derivative cones are quadratic. Derivative cones of higher degree 

better fit the underlying conic constraint, raising the prospect that the paths 
they generate lead to optimality quicker than the central path. 

u" 
o 

Cd ■ 1. Introduction 



Let £ denote a finite-dimensional Eucfidean space and let p : f — > K be a 
hyperbolic polynomial, that is, a homogeneous polynomial for which there is a 
designated direction vector e satisfying p(e) > and having the property that for 
all a; € £", the univariate polynomial t i— > p{x + te) has only real roots. Thus, p is 



^\ , "hyperbolic in direction e." 



Let A_|_+ denote the hyperbolicity cone - the connected component of {x : p{x) > 
0} containing e. Let A+ be the closure. 

^— V ■ A simple example is p{x) = J^ Xj and e — (1, . . . , 1), in which case A^.+ is the 

f^ , strictly positive orthant and A+ is the non- negative orthant. Perhaps the most 

fundamental example, however, is p{X) — det(X), where X ranges over n x n 
symmetric matrices and e = J, the identity matrix. Here, A++ is the cone of 
(strictly-)positive definite (pd) matrices and A+ is the positive semidefinite (psd) 
K> ! cone. 

C^ , Garding 16] showed for each hyperbolic polynomial p that every e € A_|__|_ is a 

hyperbolicity direction, i.e., for every x, all of the roots of t i— >■ p{x + te) are real. 
One of several remarkable corollaries Garding established is that A^+ is convex. 
(Of course A+ is thus convex, too.) (See §2 of [13] for simplified proofs.) 

The combination of convexity and rich algebraic structure make hyperbolicity 
cones promising objects for study in the context of optimization, as was first made 
evident by Giiler [7], who developed a rich theory of interior-point methods for 
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2 J. RENEGAR 

hyperbolic programs, that is, for problems of the form 

min c*x 
s.t. Ax = b 

X € A+ 

— linear programming, second-order programming and semidcfinitc programming 
being particular cases. Key to Giiler's development is that the function a; H' — In p{x) 
is a self-concordant barrier for A+; thus the general theory of Nesterov and Ne- 
mirovski [12] applies. 

A primary purpose of the present paper is to use the viewpoints provided by 
hyperbolic programming to develop a natural generalization to the notion of the 
central patllJ (a notion that lies at the heart of interior-point method theory). This 
is accomplished via derivative cones, which are direct relaxations of the underly- 
ing convex conic constraint (be it the non-negative orthant, the cone of positive 
semidefinite matrices, . . . ). 

However, perhaps more important than the "natural generalization to the no- 
tion of the central path" is our "use (of) the viewpoints provided by hyperbolic 
programming" in developing the generalization. Indeed, it is our conviction that 
even if results about hyperbolic programming never find application more general 
than linear programming and semidefinite programming, the setting of hyperbolic 
programming is favorable for engendering intriguing algorithmic ideas that other- 
wise would have been unrealized (or at least considerably delayed). 

Familiarity with the central path is not required to readily understand our re- 
sults. (The central path simply provides an initial anchor with which many readers 
are familiar.) 

The literature focusing on hyperbolic polynomials is relatively small but its 
growth is accelerating and its quality in general is distinctly impressive. Although 
the nature of our results is such that during the development we have occasion to 
cite only a few works, we take the opportunity before beginning the development to 
draw the reader's attention to the bibliography, which includes a variety of notable 
papers appearing in recent years. In particular, an appreciation of the breadth and 
quality of research ideas surrounding hyperbolic polynomials can be fostered by 
browsing [T], [2], [H], [5], and [TO]. 

2. Overviev^r of Results 

Let (/) be a univariate polynomial all of whose coefhcients are real. Between any 
two real roots of there lies, of course, a root of </>'. Consequently, because 0' is of 
degree one less than the degree of (j), a simple counting argument shows that if all 
of the roots of cj) are real, then so are all of the roots of </>'. 

In particular, if (/>(i) := p{x + te) where p is a polynomial hyperbolic in direction 
e (and where x is an arbitrary point), then all the roots of t H> <p'{t) = ^p{x + te) — 
Dp{x+te)[e] are real, where Dp{x+te) denotes the differential of p at x+te. Hence, 
the polynomial Pe{x) := Dp{x)[e\ is, like p, hyperbolic in direction e. 

For example, ii p{x) — ]^ Xj and all coordinates of e are nonzero, then p'f,{x) — 
J2i Gi Y\j^i ^j is hyperbolic in direction e. 

We refer to p'^ as the "derivative polynomial (in direction e)," and denote its 
hyperbolicity cone by A^^ g. The fact that for every x, the roots oi t >—>■ p'^{x + te) 
lie between the roots of < i— >■ p{x + te) is readily seen to imply A++ C A'^^ e ~ '^^ 
words, A'^^ e is 3- relaxation of A++ (see §4 of [13 for a full discussion). 

Of course one can in turn take the derivative in direction e of the hyperbolic 
polynomial Pg, thereby obtaining yet another polynomial - {p'g}'e{x) = -C^p(a;)[e, e] 



Central Path := {x{ri) : rj > 0} where x{t]) solves inirii rjc*x — lnp{x), s.t. Ax = b, x £ A++. 
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- hyperbolic in direction e. Letting n denote the degree of p, repeated differentiation 
in direction e resuhs in a sequence of hyperbohc polynomials 

Pe Pe^ Pe ' • ■ • i Pe ' 

where deg(pe ) = n — i. (For convenience, let pi := p.) The associated hyperbol- 
icity cones A!,!|_e - e 
of the original cone: 



icity cones A.^+^e ~ and their closures A^,* ^ - form a nested sequence of relaxations 



A, = a(°IcaWca(^)c---ca(':-^ 

The final relaxation, A"^e , is a halfspace, because pe is linear. 

The cones become tamer as additional derivatives are taken. The halfspace 
A4_"e is as tame as a cone can be, but extremely tame also is the second-order 
cone Al,."e - no cone with curvature could be nicer. As one moves along the 
nesting towards the original cone A+, the boundaries gain more and more corners. 
For example, when p{x) = Y[j ^j a-nd all coordinates of e are positive, the boundary 

dA^^'^e contains all of the non- negative orthant's faces of dimension less than n — i 
(hence lots of corners when i is small and n large). On the other hand, everywhere 
else, 9A4_ e has nice curvature properties (no corners) , as is reflected in the following 
motivational theorem pertaining to every hyperbolicity cone whose closure is regular 
(i.e., has nonempty interior and contains no subspace other than the origin). 

Theorem 1. Assume A_|_ is regular and < i < n — 2. 

(A) The intersection A+ n dA_le is independent of e E A++ 

(thus, a face of A+ which is a boundary face of A_l'e for some e G A+^. is a 
boundary face for all e G A+_,_ j. 

(B) If e E A+_^ then any boundary face o/ A\_ g either is a face o/ A^ 
or is a single ray contained in A^'+^e \ A4_% • 

We show in Sj^that the theorem is a consequence of results from [T3]. (In order to 
make the present section inviting to a broad audience, nearly all proofs are delayed.) 

Before moving to discussion of hyperbolic programs, we record a characterization 
of the derivative cones that is useful both conceptually and in proofs: 

A^'^ = {x : pi^^ (x) > for aU j = i, . . . , n - 1} . (2.1) 

(This is immediate from Proposition 18 and Theorem 20 of ilS, .) 



-f >> 



Consider a hyperbolic program 



nun ex 
s.t. Ax^b ^ HP 

a; e A+ 

and its derivative relaxations in direction e, 

HP^P (i = l,...,n-l) . 

(Strictly speaking, "min" should be replaced with "inf ," but we focus on instances 
where a minimizer exists.) The optimal values for the derivative relaxations HPj.'^ 
form a decreasing sequence in i, due to the nesting of the derivative cones. 

Let Feas (resp., Feas^*') denote the feasible region of HP (resp., HP^*-*) - the 
set of points satisfying the constraints. Let Opt (Opt).'') denote the set of optimal 
points - a.k.a. optimal solutions - and let val denote the optimal value of HP. 



inn 


ex 


s.t. 


Ax = b 




X e a1'.^ 



4 J. RENEGAR 

We assume 6^0 (thus, the origin is infeasible, and the feasible sets are not 
cones), A is surjective (i.e., onto), and c* is not in the image of A* (otherwise ev- 
ery feasible point would be optimal). 

We assume A^ is a regular cone. Then, for 1 < i < n — 2, A^^^, also is regular 
([13], Proposition 13). 

From these assumptions and Theorem [T]^B) immediately follows a fact that will 
play a critical role: 

If 1 < z < n — 2, then either 
Opt« - , 

Opt^^) = Opt , or \ (2.2) 

Opt^*' consists of a single point 

and the point is contained in relint(Feas*.*'''^-') \ Feas^'"^-* , 

where "relint" denotes relative interioio. 

Thus, for each 1 < i < n — 2, the cone A_^+ is naturally partitioned into three 
regions, one consisting of derivative directions e for which Opt^'' = Opt, a second 
consisting of directions for which Opt^*-* = 0, and the third consisting of directions 
for which Opt^*-* consists of a single point lying outside the feasible region for the 
original optimization problem HP. We associate names with this partitioning of 
A++, but before doing so, we introduce a restriction. 

We shall only be concerned with derivative directions e satisfying Ae = b (indeed, 
key arguments rely heavily on A{e — x) = for x e FeasJ,*^). Thus, the derivative 
directions we consider satisfy e € Feas|.'' for all i - in particular, HPj.*' is "strictly" 
feasible, as e e A++ C A+|^e. 

We distinguish two sets of derivative directions for < i < n — 1: 

The i^^ central swath is the set 

Swath(i) := {e e A++ : Ae ^ b and Opt^^^ ^ 0} , 
and the set of core derivative directions is defined by 
Core(i) := {e e Swath(i) : Opt^^^ = Opt} . 

When 1 < i < n — 2 and e € Swath(i) \ Core(i), we use Xe to 
denote the unique point in Opt J.*' (unique by (|2.2|) ). 

For reference, we note that from (|2.2p . 

x« e A^t^ n aA« . (2.3) 

Trivially, if Opt =^ (resp., = 0), then Swath(O) = Core(O) = relint(Feas) (resp., 
= 0), that is, the zeroth swath coincides precisely with the relative interior of HP's 
feasible region (resp., is the empty set). More interestingly, the swaths and cores 
are nested: 

Swath(O) 3 Swath(l) D • • • D Swath(n - 1) 

Core(O) D Core(l) D • • • D Core(n - 1) 

For the cores, the nesting is an easy consequence of the reverse nesting 

Feasi°) C Feasi^) C • • • C Feas^""^) . 

These reverse nesting also provide the crux in proving the nesting of the swaths, a 
proof we defer to 21 



The "relative interior" of a convex set S C £ is the interior of 5 when considered as a subset 
in the smallest affine space containing S (where the afEne space inherits the topology of the 
Euclidean space £). 
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A consequence of the nesting of swaths is that if any swath is nonempty, then 
Swath(O) ^ - equivalently, Opt ^ 0. 

Whereas a path is narrow (one-dimensional), swaths can be broad, just as 
Swath(i) typically fills much of the feasible region for HP when i is small. But 
why do we use the terminology "centra/ swaths" rather than simply "swaths"? The 
following elementary theorem (proven in 21 gives our first reason. 

Theorem 2. Swath(n — 1) = Central Path . 

The central path is fundamental in the literature on interior-point methods. The 
path leads to optimality. Most of the algorithms follow the path, either explicitly 
or implicitly. A foremost goal of the present paper is to show that not only does the 
central path lead to optimality, but all central swaths lead, in a natural manner, 
to optimality. We show, in particular, that through each point e G Swath(i) (for 
1 < i < n — 2), there is naturally defined a trajectory which leads from e to 
optimality; moreover, the trajectory remains within Swath(i) until optimality is 
reached. An intriguing possibility is that for small values of i, the trajectory might 
lead to optimality "more quickly" than the central path. (Motivation for this 
possibility will become clearer as the reader proceeds.) 

For 1 < i < n — 2, consider the idealized setting in which for derivative directions 
e £ Swath(i), an exact optimal solution for HP*.*' can be computed. If the optimal 
solution lies in A^, then clearly it lies in Opt, the set of optimal solutions for the 
original optimization problem HP. In this case our goal of solving HP has been 
accomplished. On the other hand, if the optimal solution does not lie in A+ , then 
e £ Swath(z)\Core(z), and the optimal solution is the unique point Xe in Opt^''. In 
this case how can we move towards solving HP? How can we construct a trajectory 
1 1—)- e(t) for which e(0) = e and such that either the trajectory converges to Opt or 
the path 1 1— > i^Zn converges to Opt (or both)? 

An apparently easier task would be to create a trajectory 1 1— ^ e(t) for which 1 1— >■ 
c*e(i) is monotonically decreasing. Indeed, we could define the trajectory implicitly 
according to the differential equation ;^e(i) — a;gZ^— e(t) (e(0) £ Swath(z)\Core(i)) 

that is, move from e(i) infinitesimally towards the optimal solution x\L . Assuming 
this does result in a well-defined trajectory, then clearly, t i->- c*e{t) is decreas- 
ing. However, there are no clear reasons suggesting that the trajectory t i-)- e{t) 
converges to Opt. It is conceivable, for example, that the trajectory reaches the 
boundary 9A_,. in finite time, converging to a point having better objective value 
than e(0), but not to a point in Opt. Alternatively, in finite time the trajectory 
might reach Core(i). It seems plausible that the path t i— > a;^'Z^ then would have 
limit in Opt. But how would one prove it? How does one even rule out the pos- 
sibility that in finite time, the path t i— >■ a;gZ> goes to infinity while the trajectory 
i I— >■ e(t) remains bounded but with no limit points in Opt? 

Resolving these issues, and similar ones, is our primary focus. We show that the 
differential equation ^e(i) = xlL — e(i), e(0) £ Swath(i) \ Core(i), does result in 
well-defined trajectories in Swath(i)\Core(i), and we show that either the trajectory 
t I—)- e{t) or the path t i— )■ xlL does converge to Opt. We show many other things 
as well, but to accurately explain, first we must formalize. 

In place of ^e(i) = xlL — e{t) we often write e(i) = x\L — e{t). That this dy- 
namics results in well-defined trajectories is immediate from the following theorem, 
whose (relatively routine) proof is in Sj5] 
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Theorem 3. Assume 1 < i < n — 2. The set Swath(i) \ Core(i) is open in 
the relative topolog'i 
Swath(i) \ Core(i). 



the relative topology o/ relint(Feas). Moreover, the map e i~> Xe is analytic on 



As an aside, we note that every e £ Swath(n — 2) has a unique optimal solution, 
simply due to the strict curvature of second-order cones. Thus, we can naturally 
extend the definition of Xe~ to include all derivative directions in Swath(n — 2), 
not just the ones in Swath(n— 2)\Core(n— 2). It happens that for the case i — n— 2, 
virtually all of our results remain valid when "Swath(n — 2)" is substituted for 
"Swath(n — 2) \ Core(n — 2)" (as we discuss while proving our theorems fiii}3l fT0| V 
With regards to Theorem [3] in particular, the extended map e i— >■ Xe is analytic 
on all of Swath(7T. — 2) (which is open in the relative topology of relint(Feas)). 

Theorem [3] implies that when initiated at e(0) € Swath(z)\Core(i), the dynamics 
e(t) = x^lL — e(t) results in a well-defined trajectory. The trajectory remains in 
Swath(i) \ Core(i) for all time (i.e., is defined for all < i < oo), or is defined only 
up to some finite time due either to reaching the boundary of Swath(i) \ Core(i) or 
escaping to infinity. Let r(e(0)) denote the time at which the trajectory becomes 
undefined (possibly T(e(0)) == oo). We refer to t ^ e{t) (0 < t < r(e(0))) as a 
"maximal trajectory." For brevity, we often instead write "0 < t < T" with the 
implicit understanding that the time of termination, T, depends on e(0). (Some of 
our results distinguish between T < oo and T — co, but never is distinction made 
between different finite termination times.) 

Here is the formal statement of results partially described earlier: 

Main Theorem (Part I). Assume I < i < n-2 and let t ^ e{t) {0 < t < T) 
be a maximal trajectory generated by the dynamics e{t) — xlL — e{t) beginning at 
e(0) e Swath(i)\Core(i). 

(A) The trajectory i i— > e(i) is bounded, and t i-^ c*x\L is strictly increasing, 
with val (the optimal value of HP) as the limit. 

(B) If T — oo then every limit point of the trajectory 1 1— >■ e(i) lies in Opt. 

(C) If T < oo then the trajectory t i— >■ e(i) has a unique limit point e and 
e G Core(i),' moreover, the path t i— > xlL is bounded and each of its limit 
points lies in Opt. 

The Main Theorem (Part I) is proven in fJS] 

An immediate consequence of the theorem is that T = oo whenever Core(i) = 0, 
as is the case whenever Opt n A^'j^g ^ for some (and hence, by Theorem [TJ A), 
for all) e e A++. 

Perhaps the reader wonders as to the inspiration for the idea that the trajectories 
t I— >■ e(i) arising from the dynamics e{t) = x ,.-. — e{t) lead to optimality, either in 

the limits of the trajectories themselves or in the limits of the paths 1 1— > x,\. The 
following theorem (whose proof is in ^ serves to clarify the inspiration, as well as 
to further illuminate our choice of the terminology ^^ central swaths" (as opposed to 
simply "swaths"). 

Theorem 5. Assume i ^->■ e(i) (0 < i < T) is a maximal trajectory arising from 
the dynamics e{t) = a; ",x — e{t), starting at e(0) G Swath(r7, — 2) \ Core(n — 2). 

//e(0) e Central Path then {e{t) : < i < T} C Central Path. 

We remark that when i — n^2, the termination time T always is oo, even when 
Core(n-2) 7^0 (see Eg]). 
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From the two theorems, we see that the central path is but one trajectory in a 
rich spectrum of paths. Moreover, the central path is at the far end of the spectrum, 
where the cone A_|_ is relaxed to second-order cones A\_"J ' . Second-order cones have 
nice curvature properties but are far cruder approximations to A^ than are cones 
Ay e for small i. This raises the interesting prospect that algorithms more efficient 
than interior-point methods can be devised by relying on a smaller value of i, or on 
a range of values of i in addition to i = n — 2. 

Some exploration in this vein has been made by Zinchenko Q14|.[T6]). who showed 
for linear programs satisfying standard non-degeneracy conditions that if i is chosen 
appropriately and the initial derivative direction e(0) is within a certain region, then 
a particular algorithm based on discretizing the flow e{t) — x\^^ — e{t) converges 
R-quadratically to an optimal solution. 

Before moving to the next result, we acknowledge that maybe the Main Theorem 
can be strengthened without restricting its general setting. For example, in Part 
1(B) there is no statement that limit points of the path t >—>■ a^gZ-, are optimal 
solutions for HP - there is not even a statement that the path is bounded. This 
omission seems odd given that the trajectory 1 1-^ e(t) is following the path 1 1->- x^lL 

(according to the dynamics e{t) — x^lK — e{t)) and given that the theorem states 
limit points of the trajectory are optimal for HP. Intuitively, it seems the path 
would converge to optimality and do so even more quickly than the trajectory. The 
intuition is correct for a wide variety of "non-degenerate" problems (indeed, quicker 
convergence of the path than the trajectory underlies Zinchenko's speedup), but we 
have been unable to find a proof - or counterexample - in the general setting of 
the theorem|j 

In a similar vein, maybe it is true when T — oo that the trajectory 1 1— > e{t) has 
a unique limit point. If, like the central path, the trajectory was a semialgebraic set 
then the limit point indeed would be unique (simply because every semialgebraic 
path that has a limit point has exactly one limit point). However, we doubt that the 
trajectories are necessarily semialgebraic in general, and we see no other approach to 
proving uniqueness. The theorem leaves open for the general setting the possibility 
that when T = oo (resp., T < (X)), some trajectories t i— >■ e(i) (resp., some paths 
1 1— >■ xy^^) have non-trivial limit cycles - yet we have no examples of such behavior. 

Of course the dynamics of moving e towards an optimal solution x can also be 
done for e E Core(i), in which case e would be moving towards x G OptJ,*-^ = Opt. 
As a matter of formalism, it would be nice to know that such movement would result 
in a new derivative direction for which x is still optimal, that is, a new derivative 
direction that also is in Core(i). The following theorem (proven in HlOp establishes 
a bit more. 

Theorem 6. Assume e G Core(i) and let A be the minimal affine space containing 
both e and Opt. Then 

A n A++ C Core(i) . 

In the following conjecture, the empty set is taken, by default, to be convex. 
Conjecture. Core(z) is convex. 



To gain a sense of the difficulties (and how first impressions can mislead), consider that for 
any value < T < oo, it is straightforward to define dynamics on R-^ that generates a pair of 
paths a{t), b{t) for which 6(t) = a{t) — b{t) and as t — )■ T, a(t) spirals outward to infinity whereas 
b{t) spirals inward to a point. (Thus, although a{t) is "leading" 6(t), the paths end (infinitely) far 
apart.) 
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Much work remains in order to transform the ideas captured in the Main The- 
orem (Part I) into general and efBcient algorithms. For example, devising and 
analyzing efficient methods for computing xi given e is, in the general case, a chal- 
lenging research problem. However, computing Xe (that is, the case i = n — 2) 
amounts simply to solving a least-squares problem and using the quadratic formula. 
Here, Chua [J, starting with - and extending - ideas similar to ones above, devised 
and analyzed an algorithm for semidefinite programming (and, more generally, for 
symmetric cone programming) with complexity bounds matching the best presently 
known - 0[^/n\og{l / e)) iterations to reduce the duality gap by a factor e when A^ 
is the cone of n x n sdp matrices. 

Although in the present work we do not analyze methods for efficiently computing 
Xe , we now present a few results relevant to algorithm design. These results also 
are important to the proof of the Main Theorem. 

(i) 

For the next two theorems, nothing is gained by distinguishing pe from any other 
hyperbolic polynomial, so we phrase the results simply in terms of a polynomial p 
hyperbolic in direction e, and its first derivative p'^. (The two theorems, moreover, 
do not require A+ to be a regular cone.) 

Let 

Qe := p/p'e , 
a rational function. The natural domain for qe is A^^ g, because p'^{x) > for 
X G A^^ e and p'f,{x) = for x in the boundary of A'^^ ^. 

The following result is proven in ^ 

Theorem 7. The function q^, : A^^ ^ — > M is concave. 

Previously, qe was known to be concave on the smaller cone A++ (see |T]). For 
us, the significance of the function being concave on the larger cone A^^ is that, as 
the following theorem illustrates, HP can be reformulated as a linearly-constrained 
convex optimization problem (no explicit conic constraint). For motivation, think 
of the situation where one has an approximation to an optimal solution for HP and 
the goal is to compute a better approximation. 

Theorem 8. The optimal solutions x e A'^^ ^ for HP are the same as for the 
convex optimization problem 

min^gA^^^ -\nc*{e- x) - qe{x) 
s.t. Ax — b . 

(A caution: The theorem asserts nothing about optimal solutions for HP that 
happen to lie in the intersection of the boundaries (9A+ and dA'_^ g.) The theorem 
is proven in ^ 

Since, by (|2.3p , Xe G A+^,_e for e € Swath(i) \ Core(i), the following corollary 
is immediate, except for the assertion regarding the second differential, which is 
established in ^ 

Corollary 9. // 1 < i < n — 2, e e Swath(i) \ Core(i) and 

P^\x) 



f{x) :— — lnc*(e — x) 



P^t'\x) 



then Xe is the unique optimal solution for the convex optimization problem 

TL\in^ f[x) 
s.t. Ax ~ b . 
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Moreover, D"^ f{xe )[v,v\ > if v is not a scalar multiple of Xe (in particular, if 
V ^ Q satisfies Av — Q). 

A consequence of the assertion regarding the second differential is that Newton's 
method will converge quadratically to Xe if initiated nearby. (Which is not to say 
that Newton's method is the algorithm of choice for this problem.) 

Our final result relates the dynamics e{t) ~ ^e(t) " ^('■) ^° ^^*^ optimization 
problem dual to HP: 

supj^.^s* y*b \ 

s.t. y*A + s* =c* \ HP* , 

.* e a: J 

where A* is the cone dual to A+O A pair {y* , s*) satisfying the constraints is said 
to be "strictly" feasible if s* G int(A*) (interior). 

Letting val* denote the optimal value of HP* (val* = — oo if HP* is infeasible), 
we havCjiust as a matter of tracing definitions, the standard result known as "weak 
duality"!! val* < val. 

Optimizers expect that if a dynamics provides a natural path-following frame- 
work for solving a convex optimization problem, then not only do the dynamics 
generate paths leading to (primal) optimality, also the dynamics somehow generate 
paths leading to dual optimality. 

For e e Swath(i) \ Core(z), define 



e^^ ■- ^%)^Dp^^ ^here x = x« , 



P^:^'\x) 



and where Dpe {x) is the differential of pe at x, i.e., the linear functional defined 

dtt 



on vectors v by Dpi {x)[v\ := -jjPe {x -f tv)\t=o- The following result is proven in 



Main Theorem (Part II). Assume l<i < n-2 and let t M- e(t) [Q <t <T) be a 
maximal trajectory for the dynamics e = xl —e starting at e{0) G Swath(i)\Core(i). 
Then y* A -K sV^-. — c* has a unique solution y* — J/gZ-,, and the pair (j/g/j-,, s^/O is 
strictly feasible for HP*. Moreover, 

y ,Lb — c*x ,L > val 

ye(t) e(t) f^rp 

(in fact, increases to val strictly monotonically) and the path t i— > (yg/j-ij ^gZ-)) is 
bounded. 



Consequences are, of course, that val* = val ("strong duality") and that the limit 
points of i I— >■ (2/g((\, s)l(t)) fo'"™ ^ nonempty set, each of whose elements is optimal 
for HP*. 

Thus, although the path 1 1~> x^A is infeasible for HP, and the feasible trajectory 
t I— > e(i) can potentially not converge to optimality (it might instead converge to 
e G Core(j)), there is naturally generated a path t >-?■ {y^u\T s^(lu\) that both is 
feasible for HP* and converges to optimality. 



The dual cone K*^ consists of the Unear functionals s* satisfying s*x >0 for all x £ A+ 
This is proven simply by observing that for feasible x and (y*, s*), 

c*x = (y* A + s*)x = y*b + s*x > y*b , 

the inequality due to x S A-|- and s* S A^. 
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Perhaps, then, the algorithmic framework we have posed as being for the primal 
optimization problem would be better posed as being for the dual, since by inter- 
changing the primal and the dual, there would be a single path generated for the 
primal, and that path would be both feasible and converge to optimality. 

(i) 

Lastly, we mention that in defining the sequence of derivative polynomials pe 
(i = 1, . . . , n — 1), we could have used various derivative directions ei, . . . , e„_i, 
choosing ei from the hyperbolicity cone for p and defining p'^-^{x) := Dp(x)[ei], 
choosing 62 from the (larger) hyperbolicity cone for p'^_^ and defining p"^ ^^ i^) ■= 
Dp'^^{x)[e2] — D^p{x)[ei,e2], and so on. Several results in the following pages 
can be extended to this more general setting. However, computing multidirec- 
tional derivatives can be prohibitively expensive, even for the innocuous-appearing 
hyperbolic polynomials p{x) = Y[7=i ^F-^ naturally arising from polyhedral cones 
{x € K" : afx > for all i — 1, . . . , n}; indeed, choosing ei, . . . , e„ to be the stan- 
dard basis, D"p(a;)[ei, . . . , e„] is the permanent of the matrix whose i"^ column is 
a*. 

By contrast, if the same direction e is used for all derivatives (z = 1, . . . , n— 1), the 
resulting polynomials pe (resp., their gradients, their Hessians) can be efficiently 
evaluated at any point if the initial polynomial p (resp., its gradient, its Hessian) 
can be efficiently evaluated at any point. This is straightforwardly accomplished 
by interpolation, and can be sped up via the (inverse) Discrete Fourier Transform 
(see §9 of J13j for some discussion). As a primary motivation for the present paper 
is designing efficient algorithms, it thus is sensible to restrict consideration to the 
same direction e being used for all derivatives (i = l,...,n— 1). 

3. Prelude to the Analysis 

Now we turn to proving the results. The theorems are proven in the order in 
which they were stated with the exceptions of IPart II of the Main Theorem and 
Theorem [6l The proof of the first is delayed because it depends on theorems that 
were stated later. The proof of Theorem 6 is delayed, until the end, due to the 
combination of the proof being long and the theorem being less important than 
others. 

To ease burdens on the reader, each theorem is restated before its proof, and 
is renumbered to match the section in which it is proven (in part so it is clear 
there is no circularity among the proofs). Additionally, concepts and definitions 
are recalled as they first retake center stage, and a few supplementary results are 
presented. Thus, the reader is freed from having to refer to the preceding "overview 
of results." 

Several proofs rely fundamentally on results from [13] , a paper on structural as- 
pects of hyperbolicity cones (and hyperbolic programs), a paper for which a primary 
goal was to provide a ready reference of "lower-level" details so that subsequent pa- 
pers (such as the present one) could avoid drawn-out proofs. The relevant results 
are presented in propositions or theorems at the beginning of sections where the 
results are first needed, with the exception of the following results. 

One result from J13l is used time and time again - the characterization we 
recorded as (|2.1|) . that is, 

A^',^ = {x : pi^^ (x) > for aU j = i, . . . , n - 1} . (3.1) 

For ease of reference we record the following two characterizations that are similar 
to the one above: 

Al'i.e = {x : pi^^ (x) > for all j = i, . . . , n - 1} , (3.2) 
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A^'|,e = {x : p'i^ (x) > and p'-^^ (x) > for all j = i + 1, . . . , n - 1} . (3.3) 

As was noted earlier, p.ip is immediate from Proposition 18 and Theorem 20 in 
|13j . The representations p.2p and p.3p are established in the two paragraphs 
following that theorem. 

Readily proven from the above characterizations are that for 1 < i < n — 1, 

xeA^lXAte'^ ^ pt'\x)<0 (3.4) 

and 

xedA^_il ^ pI;-^Hx)<0. (3.5) 

On a different note, recall that the fc*'' differential D^ f{x) of a function / : M" i-^ 
R is defined on fc-tuples of vectors by 

D^f{x)[vi, . . . , Wfc] := ^ • • • -^f{x + iiz;i + • • • ifeffe)|t,=o • • • k=o • 

If / is analytic - as are hyperbolic polynomials - the differential is well-defined 
for every k, and is symmetric (i.e., for all permutations tt of {1, . . . , fc}, the values 
2? /(a;)[w^(i), . . . , U7r(fc)] ^^^ identical). Moreover, the differentials are multilinear, 
that is, Vi I— >■ D^ f{x)[vi, . . . , w^.] is linear when wi, . . . , Vi_i,Vi+i, . . . , Wfc are fixed. 
Fixing vectors wi, . . . , Wj results in a symmetric, (fc — j)-multilinear form 

[vi,...,Vk~]] ^ D'' f{x)[wi, . . . ,Wj,Vi, . . . ,Vk-j] ■ 

This form is denoted D''f{x)['Wi, . . . ,Wj]. 

A fact used extensively, and which is easily proven (by, say, induction on j with 
base case j — 0), occurs when / is homogeneous of degree £ (i.e., ,f(tx) — t^f(x)): 
For 0<j <k<£, 

D^f{x)[x^_^] = (I0+j^D''-^fix) . 

j times 

In particular, for 0<j<k<n — i^ 

D^pf{x)[x, . . . ,x] = (n^lz^7^^-^p«(x) . (3.6) 



(n— j-fc)! 
j times 

A consequence of (|3.6p used occasionally is that ifO<fc<n — i, then 

D^j^(e) ^ D^+^p{e)[e^_^ = ^^i^D^p{e) (3.7) 

i times 

- in particular, D^pe {e) is a positive multiple of D''p{e). 

Finally, we mention that the nesting of the derivative cones often is used implic- 
itly. Two examples of assertions made without the nesting being mentioned: "If 
X e Opt^*) and X e Feas, then x G Opt." "If x e A<^*|,e then pi''^^\x) > 0." 

4. Proofs of Theorems 1 and 2, and the Nesting of Swaths 

The following theorem records results from [13^ that are used in this section. 
Theorem 4.1. For any polynomial hyperbolic in direction e, the following hold: 

(A) For I = 1, . . . , n - 1, A**^;^' n dA'-_il = A+ n aA^'^t 

(B) For i — 0, . . . ,n~ 1, the intersection A+ n dA^je is independent of e d A++ . 

(C) For i — 0, . . . , n — 2, the Uneality space of A)^^^ consists precisely of the 
points in A+^e H 9A4_"e 

(D) The cones A^^/g, A^^.g, . . . , A4_"e have the same Uneality space (thus, if one 
of the cones is regular, all are regular). 
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(E) If h.j_ is regular and < i < n — 2, then any face o/ A+^e either is a face of 
A+ or is an extreme direction of A_l'e . 

Proof. Result (A) is Proposition 16 in [13]; (B) is established from Theorem 12 by 
induction (with base case i = 0) and by use of Proposition 22 (which shows that 
although the definition of the sets "^'"A^i^" appearing in Theorem 12 depends on 
a derivative direction e, the sets actually are independent of e e A++); (C) follows 
from Proposition 11; (D) is immediate from Proposition 13 by induction with base 
case 1 = 1 (alternatively, follows from (A) and (C)); (E) is Proposition 24. D 

Here again is Theorem [l] but renamed: 
Corollary 4.2. Assume A+ is regular and < i < n ~ 2. 

(A) The intersection A+ n dA^'e is independent of e G A++ 

(thus, a face of A+ which is a boundary face of A4. e for some e G A+^. is a 
boundary face for all e G A^_|_j. 

(B) If e E A+^ then any boundary face of A^^^ either is a face of A^ 
or is a single ray contained in A_^^,J \ A\^^e . 

Proof. Parts (B) and (E) of Theorem 14.11 mostly prove the corollary. It remains 
only to show 

xe{dA^S)\A+ ^ xeA^:t'J\A^:-'K (4.1) 

However, Theorer d4.ir A) immediately gives 

X e (dA'^l) \ A+ = 

and upon substituting i + 1 for i, immediately gives 
that is, gives 



xe{dA^S)\A+ ^ x^A^e'^ 



X G Ai',l \ A+ ^ x(^ dA^_^y> 



a;GA«\A+ ^ xGA^:t^J. 
The implication (|4.ip is thus established, concluding the proof. D 

Throughout the remainder of the paper, our standard assumptions ap- 
ply without being made explicit in the statements of theorems, propo- 
sitions, etc. Recall these assumptions regard the objective function c*x, the 
equations Ax — b, and the cone A+: 

• b ^ [in particular, the origin is infeasible) 

• A is surjective (i.e., onto) 

• c* is not in the image of A* (otherwise all feasible points would be optimal) 

• A+ is a regular cone, that is, contains no subspaces other than {0} 

Thus, by Theorem 14. If D). A+ e is regular for all < i < n — 2. 
Recall the definitions of swaths and cores: 

Swath(i) := {e G A++ : Ae = 6 and Opt^ ^ 0} , 
Core(i) := {e G Swath(i) : Opt^^^ = Opt} . 

Recall, too, that the above corollary (aka Theorem 1) and our standard assumptions 
easily imply that ifl<i<n — 2 and e G Swath(i) \ Core(i), then Opt^*' consists 

(i) 

of a single point, denoted Xe . The corollary implies, moreover, that 

a;W e relint(Feasi*+i)) \ Feasi'-^^ . (4.2) 

In ^we noted that from the nesting 

Feasl°) C Feasl^) C • • • C Feasl"-^) 
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easily follows the nesting of cores, 

Core(O) ^ Core(l) ^■■■^ Core(n - 1) . 
We now establish (a bit more than) the nesting of swaths. 
Proposition 4.3. The swaths are nested, 

Swath(O) ^ Swath(l) 3 • • • D Swath(n - 1) 
(thus, if any swath is nonempty, so is Swath(O) - equivalently, so is OptJ. 

Moreover, i/Swath(n— 1) ^ 0, or i/Swath(i)\Core(i) ^ for some 1 < J < n—2, 
then Opt is a bounded set. 

Proof. We know Core(i) C Core(i — 1), so to prove the nesting of swaths, we assume 
e e Swath(i) \ Core(i) and show e e Swath(i — 1). 

First consider the case 1 < i < n — 2, where Opt^*' consists of the single point Xe 
- in particular, Opt^'-^ is nonempty and bounded, from which follows by standard 
convexity arguments that the level sets {x G FeasJ.*' : c*x < a} (a € K) are 
bounded. Since Feas^*~^^ C Feas^'\ the level sets {x e Feas^*""^^ : c*x < a} also 
are bounded. From this and the nonemptiness of FeaSg -* (indeed, e e FeaSg -*) 
easily follows Opt^*"^-* ^ 0, that is, e € Swath(i — 1), as desired. 

Observe, too, that the boundedness of the level sets {x € Feas^*-* : c*x < a}, 
and the relations Feas^'-* 3 Feas^-'-' for j = 0,...,i, imply Opt^-'-' is bounded for 
all j' = 0, . . . , i - in particular. Opt = Opt^"' is bounded, thereby establishing the 
final statement of the proposition for the case that Swath(i) \ Core(i) 7^ for some 
1 < i < n — 2. (We note as an aside that always, Core(O) = Swath(O), so never is 
Core(O) a proper subset of Swath(O).) 

We have left to prove that Swath(T7, — 1) C Swath(n — 2), and that if Swath(n — 
1) ^ 0, then Opt is bounded. Here we rely on the following result, which is 
immediate from (C) and (D) of Theorem 14. II : 

(F) For i = 0, . . . ,n — 2, the lineality space of A+ consists precisely of the points 

in(aAi':-iVAi:l 

Now, for every e € A++, the cone A"e is a halfspace, i.e., A^"e = {x : 
d*x > 0} for some linear functional d* . Moreover, since A_|_ is a regular cone, the 
hyperplane {x : d*x ^ 0} intersects the cone A_l^,, only at the origin (by (F) 

with i = n — 2). Consequently, the level sets {x € A"_e : d*x < a} (a e R) are 
bounded. 

Assume e G Swath(n — 1). Then, clearly, 

Opt("-i) ^{x:Ax^b and d*x = 0} , 
and the first-order conditions are satisfied; 

c* = y*A + Xd* for some A > and y* . 
Since c* is not in the image of A* (by assumption), it must be that A > 0, from 
which follows that each level set {x S A+"^ ' : Ax — b and c*x < P} is a level set 
{x : Ax — b and d*x < a} for some a (depending on /3), and hence is bounded 
(according to the conclusion of the preceding paragraph). From this easily follows 
Opt^"^^-* 7^ 0, that is, e S Swath(n — 2). It also follows, of course, that for all 
< i < n — 2, the level sets {a: G A+_e : Ax = b and c*x < /3} are bounded, and 
hence that Opt^*' is bounded - in particular. Opt = Opt^°' is bounded. D 

We close this section by restating and proving Theorem [5J 
Theorem 4.4. Swath(n — 1) = Central Path 
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Proof. The central path consists precisely of the points in {x S A^^ : Ax — b} 
which minimize friix) := r]c*x — hip(x) for some 77 > 0. As the functions 77 are 
convex, the first-order optimality conditions are sufficient as well as necessary; thus, 
the central path consists precisely of the points e satisfying 

77c* KDp{e) = y*A for some 77 > and y* ^ 

Ae:^V \ (4.3) 

e e A++ . J 

For every e £ A++, on the other hand, from the linearity of a; 1— > pe {x)^ we 
have Pe {x) — Dpi" {z)[x] for every point 2; - in particular for z — e - and 
thus 

Ai"-'^={x:i?p("-i)(e)[a;]>0}. 

Clearly, then, Opt^""^^ 7^ if and only if 

c* = AL'p^""^^(e) + w*A for some A > and w* . 

Since c* is not in the range of A* (by assumption), and since Dpi"' (e) — {n — 
1)! Dp{e) (by p.7p ). we thus have that e S Swath(n — 1) if and only if 

c* = A (n — 1)! Dp{e) + w* A for some A > and w* \ 

Ae^b \ (4.4) 

e e A++ . J 

Obviously, the conditions (|4.4p and (|4.3p are equivalent. D 



(0 



5. Proof of Theorem 3 

Having finished proving that for each e G Swath(i) \ Core(i), the set Opt 

(i) . . . 

consists of a unique point Xe , it is time to show that the differential equation 

Ae{t) = x^^^^^ ~ e(t), e(0) G Swath(i) \ Core(i) 

results in well-defined trajectories. This is immediate from Theorem 3, which we 
now restate and prove. 

Theorem 5.1. Assume 1 < i < n — 2. The set Swath(i) \ Core(i) is open in 
the relative topology of relint(Feas). Moreover, the map e 1— > Xe is analytic on 
Swath(i) \ Core(i). 

To establish the theorem, we introduce another theorem and a proposition. The 
proof of the proposition is left to the reader, as it follows entirely standard lines, 
and is primarily an application of the Implicit Function Theorem to optimality 
conditions (see, for example, §2.4 of |5l for similar results). 

Proposition 5.2. Assume / : £1 x £2 ^^^ K is analytic, where £1 and £2 are 
Euclidean spaces. For z e £2, define /^ : £1 — > M 6^/ fz{y) '= f{y,z), and consider 
for some linear functional y t-^ d*y the following family of optimization problems 
parameterized by z: 

miuy d*y , . 

s.t. /4y)>0. ^^-'^ 

For some z, assume y is a local optimum with the properties that Dfz{y) 7^ 0, 
and D"^ fz{y)[v,v] < for all v ^ satisfying Dfz{y)[v] = 0. Then there exists an 
analytic function z ^-^ y^ defined on an open neighborhood of z, and possessing the 
properties yg — y and for each z in the neighborhood, the point y^ is locally optimal 
for^EW. 
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The following theorem collects results from [13]. (Keep in mind that, as was 
emphasized in 21 our standard assumptions are always assumed to be in effect. The 
relevant assumption below is regularity of A+, but this assumption is immaterial 
for parts (A) and (B).) 

Theorem 5.3. Assume l<i<n— 1, eG A^^ and x £ (dA^'g) \ A^~g ' (equiva- 
lently, by Theorem\Jl^A), x G {dA%) \ A+j. 

(A) There exist open neighborhoods U of e, and V of x, with the property that 

eeC/ ^ Af^^r]V = {x(^V ■.p^^\x)>0} . 

(B) In a neighborhood of x, dA^ ^ is a manifold, whose tangent space at x is 
{v : Dp'i\x)[v] =0}. 

(C) Further assume i < n — 2. Then D'^pg (x)[v,v] < for all vectors v that 
both satisfy Dpg {x)[v] ~ and are not scalar multiples of x. 



Proof. Recall the characterization p.ip . that is, 

A'^+1^ = {x ■■ pi^^ [x) > for al\j = i,...,n^l}. (5.2) 

For i = n — 1, result (B) is trivial because x i— > pe (x) is a linear functional, 
and (A) is immediate with the additional observation that the linear functional 
varies continuously in e. Henceforth we assume 1 < i < n — 2. 

Theorem 14. If A*) gives 

xe(aA^S)\A+ ^ xeA^:t'J, 

and hence, 

pi^\x)>0 for j = i + l,...,n-l . (5.3) 

Thus, since the polynomials vary continuously in e as well as in x, there exist open 
neighborhoods U of e, and V of x, for which 

{e,x) eU X V => p^-'^ (x) > for aU j = i + 1, . . . , n - 1 . 
This and (|5.2p establish statement (A) of the present theorem. 

In light of (A), to establish (B) we need only show Dp^ (x) ^ 0. However, 
Dpi'\x)[e] ^p^;+^\x) > (by dESJ; hence, Dp''^\x) ^ 0. 

Finally, (C) is a restatement of Theorem 14 in iT3], with A^'^e substituted for 
A+ (the hypothesis of regularity is satisfied due to Theorem 14. If D)). D 

Proof of Theorem \ 5.11 Assume 1 < i < n — 2 and e E Swath(i) \ Core(i). 
We claim 

i^p«(a;«)[e-a;«]^0. (5.4) 

Indeed, as Xe G (9A^'^e) \ A_^^ (using (|4.2I) . the tangent space of A+^e at Xe is 
precisely {v : Dpe {xe )[v] — 0}, by Theorem l5.3r B). However, e is in the interior 
of the convex cone A^^ e, and hence the difference e — Xe cannot be in the tangent 
space. The claim is thus established. 

Let £ := {x : Ax = 0}, a Euclidean (sub)space. Define / : £ x £ ^- M by 
fiVi z) = p^Xeiy + e)' and let /^ : £ ^- R be the function f^{y) := f{y, z). 

Let d* be the projection of c* onto £ (i.e., the linear functional on £ satisfying 
d*y — c*y for all y G £). For any x satisfying Ax — b, the projection of Dpc (x) 
onto £ is DfQ{y) where y — x — e. In particular, for all v £ £, 

Dp'i^ (xW ) [v] = Dfo{y) [v] where y := a;« - e . 
Since e - x'i^ e £, ((Ol) thus implies Dfo{y) ^ 0. 
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Similarly, for all v Cz £, 

Thus, since xi ^ £ (because 6 ^ 0, by assumption). Theorem l5.3f C) implies 
D^fomv, v] < for aWO^veS satisfying Dfo{yM = 0. 

In all, the hypotheses of Proposition 15.21 are satisfied for z = and y. Letting 
^ I— ^ 2/z be the analytic function whose existence is ensured by the proposition, it 
is readily apparent that y^ + e is locally optimal for 

miuj, c*x 
s.t. Ax = b 

p'zU^) > . 

Since z i— >■ j/z is continuous (it's even analytic), it thus follows from Theorem l5.3f A) 
that for z in an open neighborhood of the origin in £, the point y^ + e is locally 
optimal - and hence globally optimal - for the convex optimization problem HP^*|g 
- that is, j/z + e S Opt^4.g, and so z + e & Swath(i). 



However, for z in a possibly smaller open neighborhood of the origin, j/^ + e ^ A 



+ j 



because yo + e ^ Xe ^ A^ and z i— > y^ is continuous. Consequently, for z in this 
open neighborhood of the origin, Opt^*|g \ A+ 7^ 0, and hence, e ^ Core(i); thus, 
z £ Swath(i) \ Core(«), and clearly, x^^^ — yz + e. 

As e was an arbitrary point in Swath(i) \ Core(i), the proof is complete. D 

In closing this section, we recall that due simply to the strict curvature of (regu- 
lar) second order cones, for every e € Swath(n— 2) there is a unique optimal solution 
of HPJ."~ ' , thus naturally extending the map e 1— >■ xi" to all of Swath(n — 2). 
Additionally, every e g A_,.+ and every ^ x € dA_^g ' satisfies properties (A), (B) 
and (C) of Theorem 15.31 when i — n — 2 (regardless of whether x satisfies the theo- 
rem's hypothesis x ^ A^). Consequently, the proof of Theorem 15 . 1 1 easilv is made to 

be a proof showing that the extended map e 1— >■ xj^ is analytic on Swath(n — 2) 
(and Swath(n — 2) is open in the relative topology of relint(Feas)). 

6. Proof of Theorem 5 

We have been proving the theorems of f|2] in the order they appeared, and have 
just finished establishing Theorem 3, that is, have just finished establishing the 
well-definedness of the trajectories arising from the differential equation 

Ae(t) = x["l^ - e(i), e(0) G Swath(i) \ Core(i) . 

To continue proving theorems in the order they appeared, we would next prove 
Part I of the Main Theorem, which states that either the trajectory t i~-> e{t) or 
the path t H> xlL converges to optimality (perhaps both). However, there is 
groundwork to be laid before proving that result, so we jump to the one stated 
after it, a motivational theorem pertaining to the trajectories in the special case 
i ^ n — 2. 

For any e, the polynomial x 1— >■ pe (x) is quadratic, and hence for e £ A++, 
A+^e is a second-order cone, regularity due to Theorem 14. IT D). 

Recall we say that t H> e(t) (0 < t < T) is a "maximal trajectory" for the 
dynamics e{t) = x["^^^ - e{t) if e(0) £ Swath(i) \ Core(i) and T {= T{e{0))) is 
the time when the trajectory either reaches the boundary of Swath(z) \ Core(z) or 
escapes to infinity. 

Following is a restatement and proof of Theorem [51 whose only purposes in ^H 
were to motivate the choice of terminology "ceniraZ swaths" and to give insight into 
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the origin of the idea that in general the trajectories t H> e{t) (or paths t i-> x\L) 
converge to optimaUty. After completing the paper, the reader likely will not have 
much difficulty in making the statement of the theorem more completeO 

Theorem 6.1. Assume t H- e(i) (0 < t < T) is a maximal trajectory arising from 
the dynamics e{t) — x\^s — e{t), starting at e(0) G Swath(n — 2) \ Core(n — 2). 

//e(0) e Central Path then {e(t) : < t < T} C Central Path. 

Proof. The proof makes use of results known to anyone familiar with the interior- 
point method literature. 

The central path is the set {z{ri) : 77 > 0}, where z{r]) solves 

min^, T] c*x — Inp^x) 
s.t. Ax = b 

X e A++. 

In the terminology of Nesterov and Nemirovski [12j . the barrier function x 1— >■ 
— \np{x) is self-concordant, a fact whose implications were first developed by Giiler 

m- 

The barrier function is strictly convex on its domain, A+^. ("strictly" because A+ 
is a regular cone). Hence, for each ry > 0, there is at most one optimizer, z{r)). It is 
well-known from interior-point method theory that if the optimizer z(r]) exists for 
one positive value of ry, then an optimizer z(r]) exists for all ry > 0, and the path 
rj I— > z(ri) is analytic (using that a; 1— > — Inp(x) is analytic). 

To prove the theorem, it suffices to show that if z{r]) lies in Swath(i) \ Core(i) 
(which we know by Theorem lS.ll to be open in the relative topology of relint(Feas)), 
then z{ri) - the tangent vector to the central path at 2(77) - is a positive multiple 
of the difference a: ? s — 2(77). Indeed, this is sufficient to ensure that if ^(770) G 
Swath(i) \ Core(i), then as r/ increases from 7/0, the central path will remain in 
{e(i) : < t < T{z{rio))} - where e(0) = z{rio) - until reaching the boundary 
of Swath(i) \ Core(i), that is, will remain in the path of the trajectory until the 
trajectory ends, implying {e(i) : < t < T{z{rio))} C Central Path. 

Now we begin proving that 2(77) is indeed a positive multiple of x^ s — z{ri). 

Assuming the central path exists, for each 77 > there exists 2/(77)* which together 
with 0(77) satisfies the first-order optimality condition 

ric* = ^;^^Dp{z{r^))+y{^yA. (6.1) 

It is well known that if A is surjective (one of our standard assumptions), then 
77(77)* is unique, and 7/ 1— > 7/(7;)* is analytic (using that a; 1— > — lnp(a;) is analytic). 



More specifically, after understanding the paper the reader likely will not have much difficulty 
in showing the following, in which 

Central Path = {z{ri) : q > 0} , 
ECentralPath = {z{rj) ■.■q&R} 

("extended central path"), and z{ri) solves minj, ric*x — Inp(x), s.t.. Ax = b, x £ A^.+ : 

• Central Path C Swath(n - 2) 

• If e(0) e ECentralPath n (Swath(n - 2) \ Core(n - 2)), then T(e(0)) = 00 and 

{e(t) ; < t < 00} = {e e ECentralPath : c*e < c*e(0)} . 

• If e e ECentral Path n Core(ra - 2) then 

Core(n - 2) = £ n A++ = ECentralPath , 
where £ is the line containing e and the (unique) point in Optji" = Opt. 
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Differentiating in rj provides equations satisfied by z{ri), the tangent vector to 
the central path: 

^* = r4)) ^'P(^('?))[^(^)] - ^^I^SSf^^M^(^)) + yivTA . (6.2) 

It is well known, moreover, that c*z{ri) < 0. 

Using (I6.ip to substitute for Dp{z{rj)) in (16.21) . and relying on Az{ri) = 0, yields 

7(77)0* = D^p{z{7]))[z{r])] + wi-nYA (6.3) 

for some constant 7(77) and vector w{ri)* . We aim to use this condition to show 
that if z(ri) G Swath(n — 2) \ Core(n — 2), then 2(77) is a positive multiple of 

(n-2 

<V) 
{n-2 



x^ •. — z(ry). To accomplish this aim, we need an appropriate characterization of 
"" , the optimal solution to HP 



Now, for every e £ A++, it holds that A+^e n9A+,e = {0} (by regularity, and 
parts C and D of Theorem [H]). Thus, Feas^""^^ n ^A^'^^^ = ^ (because b ^ 0) 
and, as a consequence, 

xeFeas^,"-^) ^ pi^'-^^x) > (6.4) 

<^ Dpi''-^\x)[e] > 

^Dpi"-^'>{x)^0. (6.5) 

On the other hand, by the characterization (13. ip applied to A4_"e , 

Feasi"-^) ={x:Ax^b, pi"'^^ {x) > and pi"'^^ (x) > 0} . (6.6) 

It follows from (16.41) . (16.51) and (16.61) that necessary and sufhcient conditions for a 
point X to be optimal for the convex optimization problem HP),"~ ' are 



7C* = Dpi"- ^^ (x) +y*A for some 7 > and y* 
Ax = b 

X e 9a1"-') 



^ (6.7) 



Our goal is to show i is a positive multiple of Xz — z - equivalently, to show 
Xz"~ ' = z + tzioT some t > - equivalently, to show for some t > that x — z + tz 



Assume 2(77) G Swath(«) \ Core(j). For brevity, write z for z{r]) and z for z{r]). 
ur goal is to show i is a positive multi 
"~ ' = z + tz for some t > - equivaleni 
satisfies the conditions (|6.7p when e = z. 

To fix the value of t, consider the condition x G dA\^_^ '. To see that there 
exists i > for which a; = z + ti is in the boundary, begin by recalling c* z < 0, 
and hence by p.ip . Dp{z)[z] < 0. However, by p.7p . Dp{z) is a positive multiple 
of Dpz {z) - thus, Dpz {z)[z\ < . Since x (-> pz (x) is linear and 
Pz~ (z) > 0, it follows that for some i > 0, the point z + tz hes in i9A^"7 ■ By 
the nesting of cones, for a (smaller) value i > 0, the point z + tz lies in 9A\_"z . 
Consider the value of t to be fixed thusly. 

Of course x ^ z + tz satisfies Ax = b. To complete the proof, it remains to show 
there exist 7 > and y* satisfying 

-fc* ^Dp'i'-^'>{z + tz) + y*A. (6.8) 

We claim, however, there is no need to be concerned with the sign of 7, because 
so long as (|6.8p is satisfied, 7 is forced to be positive. Indeed, if (|6.8p held with 
7 < 0, then z + tz would satisfy the (sufficient as well as necessary) optimality 
conditions for the problem obtained from HP^"~^-* by replacing "minc*a;" with 
"maxc*a;", contradicting that c*z > c*{z + tz). And if (|6.8p held with 7 = 0, then 
by Theorem 15. 3r B). {x : Ax — 0} would be a subspace of the tangent space of 
dhr^^z sX z + tz - but this would lead to a contradiction, as z is in the interior of 
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A;^"^ and Az — 0. Thus, we need only be concerned with showing there exist 7 
and y* satisfying (16.81) . and not be concerned with the sign of 7. 

By linearity of a; i~>- Dpz (x) (as pj^ is a quadratic polynomial) , 
Dp^i'~^\z + tz) = Dp'i'-^\z) + tDp^^-^\z) . 

Since Dpz~ \z) is a positive multiple oi Dp{z) (by (|3.7p ). and since Dp{z) satisfies 
()6.ip . to show that x — e + tz satisfies ()6.8p for some 7 and y*, it thus suffices to 
show 

DPe''~'^Hz) = 7c* + y*A for some 7 and y* . (6.9) 

Now, for any vectors u and v, 
^2. 



^/r^Hu)[v]^i[Dp(r'\u+tv)) 



t=0 



In particular 
and hence, by (|3 



: L'pI"-^^ (w) (by linearity of a; h-> Dp'^J'^^^ (x)) 
Dpi"-'\z) ^ D^pi--'\z)[z] , 



Dp{"^ ^^(i) is a positive multiple of D'^p{z)[z] . 

Consequently, that 7 and y* can be chosen to satisfy (16. 8p follows from (j6.3p . thus 
completing the proof. D 

7. Proofs of Theorems 7 and 8 

We now consider the optimization problems HP^*-* for all < i < n — 1. The 
purpose of the present section is to establish a useful characterization of those 
optimal solutions of HP^*' which do not lie in (9A+% ■ For e G Swath(i) \ Core(i), 
the characterization precisely identifies the unique optimal solution Xe (because 
Xe ^ 9A+^t by (|4.2p ). Although only the case e G Swath(i) \ Core(i) is relevant 
to the Main Theorem, we record the characterization generally, because it has the 
potential for computational relevance also when e £ Core(j). 

In this section, the only properties used of the polynomials pe is that they are 
hyperbolic and nonconstant. Thus, to ease notation, for this section we let 
p be any nonconstant polynomial which is hyperbolic in direction e. We 

let Pg denote the derivative polynomial. 

If p is linear, then p'^ ^ is a constant polynomial and its hyperbolicity cone 
A'^^ is the entire Euclidean space £. 

To aid the reader's intuition, the following proposition (not used in the sequel) 
explains which points in Opt can possibly be not in dA'^ g. 

Proposition 7.1. If e ^ A_^__^_ then 

either Opt C ^A'^ ^ or Opt \ dA'_^,, = rehnt(Opt) . 

Proof. The proposition is trivially true when p is linear, because then, dA'^ e — ^ 
and relint(Opt) — Opt (due to Opt being an affine space). The proposition also is 
trivially true if Opt is the empty set or consists of a single point. Thus, assume 
deg(p) > 2, assume Opt has more than one point, and assume Opt is not contained 
in 5A;_g. 

Since Opt \ dA'^ e^^ (by assumption) , from the containment Opt C A'^ ^ , and 
from the convexity of the sets Opt and A'^ g, follows by standard arguments that 

rehnt(Opt) n SA'^ ^ = . 
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Thus, fixing optimal x not contained in relint(Opt), it remains to show x € dA'^ g, 
that is, to show Pe(x) — 0. 

Choose y £ relint(Opt), and let y(t) := x+t{y—x) (t e R). Clearly, iiy{t) g Feas 
then y{t) S Opt. Thus, since x is in the relative boundary of Opt, it holds that 
y{t) ^ A+ for t < 0. 

Since the line segment with endpoints x and y is contained in 9A^, we have 
p{y{i)) — for < t < 1. Since the only univariate polynomial with infinitely 
many roots is the polynomial that is identically zero, it follows p(y(t)) = for all 
t e M. Hence, from the characterizations 

A^':^ = {z : pi'\z) > for aU j = /c, . . . ,n - 1} , 

and the fact that y{t) ^ A+ for t < 0, follows p'^{y{Q)) = 0, that is, p'^ix) = 0, as 
desired. D 

For e e A++, 

define qe : A^^g -^ M by qe{x) -.^ p{x) / p'^{x) . 

As already mentioned, the purpose of this section is to develop a useful characteri- 
zation of the set Opt \ 9A'^ ^. The characterization is that the points in the set are 
precisely the optimal solutions to the following linearly constrained optimization 
problem: 

minj,gA^^^ -\nc*{e-x)-qe{x) 

s.t. Ax^b . ^ ' ' 

Critical to achieving the characterization (and critical to characterization's rele- 
vance for computation) is Theorem [7l now restated and proved. 

Theorem 7.2. The rational Junction q^ : A'^^ ^ — > M is concave. 

Proof. Introduce a new variable t and let P{x,t) :— tp(x), a polynomial that is 
easily seen to be hyperbolic in direction E — (e, 1). Let K' be the hyperbolicity 
cone for the derivative polynomial P^ - thus, K' is the connected component of 
S := {{x,t) : p{x) +tp'^{x) > 0} containing E. We claim K' is precisely the interior 
of the epigraph for —q^ : A'^^ ^ ^ M. Since K' , being a hyperbolicity cone, is 
convex, establishing the claim will establish the theorem. 

li x G dA'^ g then p'^{x) = and, according to p.Sp . p{x) < 0, so for no t do we 
have {x,t) G K'. Thus, now assuming x G A'^^ g and t > —qe{x), to establish the 
claim it suffices to show there is a path in S from {x,t) to E = (e,l) (because that 
will imply (x, t) and E are in the same connected component of S). 

Choose t satisfying t > max{— ge(sx -I- (1 — s)e) : < s < 1} - the maximum 
exists because the line segment connecting a; to e is contained in the convex set 
A^^ g and because p'^ is positive everywhere in A'^^ g. It is easily verified that a 
path in S from (x, t) to (e, 1) is obtained with three line segments; the line segment 
between (a;,t) and (x,f), the line segment between (x,i) and (e,f), and the line 
segment between (e, t) and (e, 1). D 

We now restate and prove Theorem [51 the main result of this section. 
Theorem 7.3. If e e A++ then 

Opt \ dA^ ^ — {x : X is optimal for ([7.i[ )| 
(possibly the empty set). 

Proof. For x in A'^^ g we have Dp{x) ^ (because < p'^{x) = Dp{x)[e\). Thus, 
from the characterization 

(aA+) \ M; g = {x : p{x) = and jl^'> (x) > for all j = 1, . . . , n - 1} 
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(by p.3p ). it follows that for the convex opthTiization problem HP, 

a; e Opt \ 9A;^, 

Ac* — Dp{x) + y*A for some A > and y* 

Ax = h 

X G [dK) \ dA'^^, . 

Observe that these conditions and homogeneity of p give 

Ac*(e — x) — Dp{x)[e — x] ^ p'ei^) ~ np{x) — p'^{x) , 

that is, 

c*(e — x) 
Clearly, then, 

a; e Opt \ aA;^ 

j'il-l) c* = Dp{x) + y*A for some y* 

Ax = b 

X e (dA^) \ 9A;,, 



(7.2) 



On the other hand, necessary and sufficient conditions for x to solve the convex 
optimization problem (j7.ll) are 

cUe-x) '^* ~ ^1e{x) = w* A for somc w* 1 

Ax = b \ (7.3) 

Observe that these conditions along with homogeneity of p and p'^ give 
1 = ../ , c*(e-x) 

c [e—x] ^ ' 

= Dqe{x)[e - x] 

= ^ (^p(-)[^ - -] - #Sy^Pe(-)[e - -]) 
= 1 _ z(^ ("i . ^"(^) 



Consequently, as x e A'^^ g implies Pe(a;) > and p"(x) > 0, it must be for optimal 
X that p{x) = 0. Hence, in the necessary and sufficient optimality conditions (j7.3p . 
the containment x £ A'_^_^ ^ can be replaced by a: G (5A+) \ dA'_^ ^. Since for a; 
satisfying p{x) = we also have Dq^{x) — ,\\ Dp{x)^ we see that the conditions 
(|7.3p are equivalent to 

Ax^b } (7.4) 

a; e (aA^) \ aA'^,, 



The optimality conditions ()7.4p and (|7.2p are identical. D 

We close this section with a restatement of Corollary [9] 
Corollary 7.4. Ifl<i<n-2,e£ Swath(i) \ Core(i) and 

pi'\x) 



f{x) := — In c* (e — a;) 



P^^)(x) 
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then Xe is the unique optimal solution for the convex optimization problem 

mill;,; f{x) 
s.t. Ax = b . 

Moreover, D'^ f{xe )[v,v] > if v is not a scalar multiple of Xe (in particular, if 
V ^ satisfies Av = 0). 

Proof. The first statement is immediate from. Theorein l7.3l Tlie statement regard- 
ing D^f is immediate from Tlieorem 15.3( 0. D 



As was noted in fj2l the coroUary's statement regarding D^f impHes quadratic 
invergence of Newt( 
in the present paper. 



convergence of Newton's method when initiated near to Xe ■ This is not pursued 



8. Proof of Part I of Main Theorem 

RecaU that for the differential equation 

^e(i) = a;^^,) - e(i) , e(0) = e G Swath(i) \ Core(i) , 

we let T = T(e(0)) denote the time (possibly T = oo) at which the trajectory 
terminates due either to reaching the boundary of Swath(i) \ Core(i) or escaping 
to infinity. We say that 1 1-^ e{t) (0 < t < T) is a "maximal trajectory." 

The primary goal of this section is to prove that the path t i— > e(<), or the 
trajectory t >->■ x\L (perhaps both), converges to optimality as i — > T. 

As we shall see, the proof is reasonably straightforward when T ~ oo, in which 
case the bounded trajectory 1 1— >■ e{t) has all limit points in Opt. 

Perhaps deserving of mention is that for all e G Swath(n — 2) \ Core(n — 2), it 
holds T{e) = oo, and so our reasonably straightforward proof applies. 

Unfortunately, the case T < oo is an entirely different matter. Here we show 
1 1— >■ e{t) converges to a unique point in Core(i), and we show the path 1 1— > xlL is 
bounded, with all limit points in Opt. This proof is subtle and long. It would be 
great if a more direct proof were discovered. 

Before restating the theorem and beginning the proof, we introduce a new use 
of differentials that plays a significant role in this section and later. 

As was recalled for the reader in [j3l the j*'* differential at x for an analytic func- 
tion / is the symmetric multilinear form defined on j-tuples of vectors [vi, . . . , Vj] 
by 

D^f{x)[vi,...,Vj] := ^ ■ ■ ■ -^f{x + tivi + ■ ■ ■ tjVj)\t^=Q ■ ■ ■ \t,=o ■ 

Until now, significant roles have been played only by the differentials D^pi (x) of 
the hyperbolic polynomials x t-^ pe (x). Here, e is viewed as fixed, and x as the 
variable. 

In this section, we sometimes need to view e as a variable, not just x. We need 
to differentiate once with respect to e, and j times with respect to x. For this we 
introduce the notation De{D^pe (x)) to represent the form that assigns to pairs of 
tuples [u] and [vi, . . . ,Vj] the value 

£{D^P^:UX^)[vi,...,Vj])\s=o- 
(Here we differentiated at e in direction u only after differentiating at x in directions 
vi, . . . ,Vj, but the order of differentiation is immaterial, because (e, x) >—>■ pi {x) is 
a polynomial (in particular, is analytic)). Thus, there is good reason to also denote 

the form by, say, D^Depi (x)).) 
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When acting on a specific pair of tuples [u] and [vi, . . . ,Vj], we denote tlie as- 
signed value by DeiD^Pe ix)[vi, . . . ,Vj])[u] (or by D^ {Depi {x)[u])[vi, . . . ,Vj]). 
The form is multilinear in u, wi, . . . , Vj, and is symmetric in vi, . . . ,Vj. 

The subscript on "Z?e" does not refer to a specific hyperbolicity direction, but 
rather, to the derivative being taken with respect to the hyperbolicity direction. 
Thus, for example, De{D^p ,L{x)) is the form that assigns to pairs [u] and [vi, . . . ,Vj] 

the value £{D^pi%+sui^)[vu ■ ■ ■ ,Wj])U=o- 

We have occasion to fix the direction u, in which case results a form on tuples 
[wi, . . . , Wj]; specifically, 

[vi, ...,Vj]^ D^{D^p^^\x)[vi, . . .,Vj])[u] , 

a form we denote by De{D^ pe {x))[u]. Similarly, ii vi, . . . ,Vi and u are fixed, we 
use De{D^pe {x)[vi^ . . . , Wj])M to denote the form 

[wi,...,Wj-i] ^ De{D^ p^^\x)[vi, . . . ,Vi,Wi, . . . ,Wj-i])[u] . 

Time and again we rely on a relation between the forms De{D^pe (x)) and 
£)i+ip(fc-i)(2,) whenfc> 1: 

D,{D^pi''\x)) = D,{D=+^p{x)[e, . . . , e]) 

A: times 

— k D-^ p{x)[ e, . . . , e ] (by multilinearity and symmetry) 

k ~ 1 times 

= kDi+^p(^-^\x) . (8.1) 

That is, for any e and tuples [u] and [t^i, . . . , Vj\, 

D,{D^pf\x)[vu ■ ■■,v,]M = kD^+^pf-^\x)[u,v^, . . .,v,] 

(a consequence being that the form D,. [D^pg (x)) is symmetric in u,vi, . . . ,Vj, not 
just in vi, . . . ,Vj). 

Here is the restatement of the result to which this section is devoted, IPart II of 
the Main Theorem: 

Theorem 8.1. Assume 1 < i < n — 2 and let t i-^ e{t) {0 < t < T) be a maximal 
trajectory generated by the dynamics e{t) — xlL—e{t) beginning at e(0) € Swath(i)\ 
Core(i). 

(A) The trajectory t i— > e{t) is bounded, and t M- c*x ,.-. is strictly increasing, 
with val (the optimal value of HP) as the limit. 

(B) If T = oo then every limit point of the trajectory i i— )> e(i) lies in Opt. 

(C) If T < oo then the trajectory t i— >■ eit) has a unique limit point e and 
e S Core(i),- moreover, the path t H^ xV^^ is bounded and each of its limit 
points lies in Opt. 

8.1. The case T = oo. As mentioned, our proof of Theorem 18.11 when T = oo is 
much easier than when T < oo. In this subsection, we focus on the case T — oo, 
although the following proposition is important also to the case T < oo. 



Proposition 8.2. Under the hypotheses of Theorem \8.1{ the trajectory t i~^> e(i) is 
bounded, and 1 1— >■ c*xlL is strictly increasing. 
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Proof. Since e(0) G Swath(i) \ Core(z), the set Opt|,V„s is bounded (indeed, consists 

of the single point a;|,!L), and hence so are the level sets {x <E Feas^tL : c*x < a} 

(a eM.). As Feas C Feas^.L, the level sets {a; € Feas : c*x < a} are bounded, too. 
Hence, since the trajectory 1 1— >■ e{t) remains in Feas, and since (clearly) 1 1— ^ c*e(t) 
is decreasing (strictly monotonically) , the trajectory lies entirely within a bounded 
region, that is, the trajectory is bounded. 

It remains to prove that t t-^ (^*^e(t) ^^ strictly increasing. To ease notation, let 

x{t) := x':'^ 



e(ty 

From t H^ p'V^Ax{t)) = we find that 



(i) 
\{t) 

d Ji) 



= Dp%^{x{t))[x{t)] + D,{p%^{x{t))[e{t)] 
= Dp^^l^{x{t))[x{t)] + iDpt^\x{t))[e{t)] (using dUD) 
= Dj^^{x(t))\x(i:}\ +^{{n-^ + l)p%^\x{t)) - p%^{x{t))) , 
where the final equality is due to e{t) = x{t) — e{t) and, by (|3.6p . 
Dp^:-I\xit))[xit)] = (n - ^ + 1^7/^(x(i)) . 

Thus, since p['^^^^{x{t)) = and p['^^^^\x{t)) < (by dSH)), we have 

Dp^:l^{x{t))[x{t)] > . (8.2) 

On the other hand, x{t) satisfies the first-order condition 

\{t)c* = Dp['^^^{x{t)) + y{t)*A for some A(i) > and y(i)* . 

Applying both sides to i(i), using Ax{t) = and substituting (|8.2p . shows c*x{t) > 
0. Thus, 1 1~^ c*x{t) is strictly increasing. D 

We are now in position to easily prove Theorem 18.11 when T = oo. 
Theorem 8.3. Under the hypotheses of Theorem \8.1[ if T = oo then 
• i I— > e{t) is bounded, with limit points lying in Opt, and 

(i) 



• i I— > c*x ,'•. increases strictly monotonically to val, the optimal value o/HP. 



Proof. Due to Proposition 18.21 it remains only to prove that both t i— >■ c*e{t) and 
t H> c*x%s converge to val as i — >■ oo. However, the convergence of c*e{t) to val is 
immediate from 

T = oo, c*xf^^^^ < val < c*e{t) and e{t) = xf^^^ - e{t) . (8.3) 

Due to the monotonicity of t i— >■ c*xlL (by Proposition 18. 2[) . the convergence of 

'^*^e(t) t<^ ^^^ ^^^'^ i^ immediate from (|8.3I) (indeed, otherwise at some time there 
would occur c*e{t) < val, contradicting e{t) G Feas). D 

8.2. The case T < oo. For the challenging case T < oo, we split most of the 
analysis into two propositions. 

Proposition 8.4. Under the hypotheses of Theorem \8.1[ if T < cx), then letting 
x(t) := xlL, either 

(A) The trajectory in- e{t) has a unique limit point, the limit point is contained 
in Core(i), and the path t h- > x{t) is bounded, with all limit points contained 
in Opt. 
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(B) It holds that liminit^T \\x{t)\\ = oo. Moreover, there exists a constant 
a < such that for all < t < T, 

p%;Hxit))<a\\xit)\r^+'. 

Proposition 8.5. Under the hypotheses of Theorem \8.1\ and regardless of whether 
T is finite, letting x{t) := ^e(t)! /'"" all Q < t < T we have 



d ,„ (-piy^(^W))"" ^^ pg7/Hxft)) ^^ p%'\x{t)) 



where 

Ci^i{n-i + lf, 0-2 = {i-l){n-i)(n-i + 2) and C3 = 2n - i + 1 
(and where for the case i — 1 we define p],r^\ = 0/ 

Before proving the two propositions, we show how they complete the proof of 
Theorem EH 



Theorem 8.6. Under the hypotheses of Theorem \8.1\ if T < 00 then 

• f I— > e{t) has a unique limit point, the limit point is contained in Core(i), 
and 



• 



t i~> Xf^u) is bounded, having all limit points contained in Opt; 



moreover, t 1— > c*x /^ increases strictly monotonically (to val, the optimal 



value of HP). 

Proof. We aheady know t i— > c*x,\ is strictly increasing. Completing the proof 
thus amounts to showing case (A) of Proposition 18.41 holds. Hence, it suffices to 
assume that case (B) of Proposition l8.4l holds. and then show Proposition l8.5l vields 
a contradiction. Thus, assume case (B) holds. 

Since the trajectory t 1— )■ e{t) is bounded (by Proposition 18. 2p . we can choose a 
compact set K containing the entire trajectory (we assume nothing about K other 
than it is compact). For j = i — 2,i — l,i + 1, define 

7j :— max{|pg (x)| : e £ K and ||a;|| = 1} 
(let 7i_2 '■= when i — 1). Since p^ is homogeneous of degree n — j, we have 
\p[^\x)\ < 7j Wxp-^ for ah e e i^ and aU x . 
Let x{t) :=a;^(j). 
Using the assumed bound pl^-, {x(t)) < a||x(i)||"~*"'"^ - keep in mind especially 

that a is negative - and using the positivity of pl^/f.) (a;(i)) (according to (|4.2p ). 
from the inequality of Proposition 18. 51 we find for all < t < T, 

I In ""-''^^ ^'^fl,, < C. — Mt)r + C. ^ ||x(.)|| + C3 . 
Hence, there exists a value r for which 

d (-p%)\<t))T~^ 

dt ( {i+i)i ,,-.-C\ ^ 
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Since lim inf t_j.7- ||a:(f)|| = oo (by assumption), it follows that 



However, 



S^P -, , n-i+l < °° 



-V%^\x{t))) {\a\\\x(t)\ 



1 iui II J^li-;il 



n — z— 1 II ^ '^11 ' 

a contradiction, thus concluding the proof of the theorem (except for proving the 
two propositions). D 

8.2.1. Proving the first of the two propositions. Now we begin proving Propo- 
sition |S?11 The proof relies on three lemmas. 

Lemma 8.7. Assume Opt is a bounded set and 1 < i < n — 2. Let {cj} be a 
sequence of derivative directions converging to e E ^++, and assume Xj € Opt).'/. 



oo. 



(A) // Xj -!• X, then x G Opt^'-' . 

(B) If {xj^ is an unbounded set, then Opt).'' = and liminf ||a; 

(C) // {xj} is an unbounded set and limsupc*a;j > — oo, then x.j/\\xj\\ has 
exactly one limit point d; moreover, d satisfies pe (d) < 0. 

Proof. First assume Xj — > x. Then pi. (x) = limpej (xj) for all k. However, by 
^^,pi'^\xj) > Oforall/c = i,...,n-l. Thus, pi''^(a;) > for all fc = i, . . . ,n- 1. 
Hence, again invoking p.ip . x € ^+,e- Since, trivially. Ax = b, we thus see that 
X e Feas^'^. 

To prove that x not only is feasible but is optimal, we assume otherwise and 
obtain a contradiction. Thus, assume x G Feas^,*-' satisfies c*x < c*x. By nudging a; 
towards the strictly feasible point e, we may assume x is strictly feasible in addition 
to satisfying c*x < c*x. However, by (|3.2p . strict feasibility of x implies pi (x) > 
for all k = i, . . . ,n — 1. Since pi/ {x) -^ pi (x), we thus have, again using (|3.1I) . 
that X € Feasi^' for j > J (some J). But then 

c*x > c*x > limc*a;j — c*x , 
a contradiction. Hence, x is optimal, and assertion (A) of the lemma is established. 

Now assume {xj} is unbounded. Choose a subsequence {xj^} for which 
liminf£_j.oo lla^jjll = oo and let d be a limit point of {2:j>/||a;j"f ||}- Then d is a 
feasible direction for HP),''', that is, satisfies Ad = and d e A^'^- (That Ad — 
is trivial. That d e A^'e follows from pi (d) = liTae^oo pij {xj^ /\\xji\\) > for 
k = i, . . .n— 1, the inequality being due to Xj^ € A\^ e- along with homogeneity of 
Pe .) The feasible direction d satisfies c*d < (because e £ Feas; ■* - and hence 
limsup£_j.o^ c*Xjf < oo ~ and because liminf^^oo ll^^jJI = oo). 

Clearly, now, if Opt).'^ ^ then Opt),'^ is unbounded (in direction d), implying 
by Corollarv l4.2f B) that Opt),'-* — Opt. But this would contradict our assumption 
that Opt is bounded. Thus, Opt^'^ = 0. 

To conclude the proof of assertion (B) of the lemma, it remains to show lim inf \\xj\\ 
oo. But if this identity did not hold then {xj} would have a limit point, and thus. 
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by assertion (A) of the lemma, we would have Opt^*-* ^ 0, contradicting what we 
just proved. The proof of assertion (B) of the lemma is now complete. 

Now we prove assertion (C). Assume {xj} is unbounded and limsupc*^;^ > — oo. 

With {xj} unbounded, we already know from above that every limit point d of 

{xj/llxjil} is a feasible direction for HP^*-' (that is, satisfies Ad = and d S A4_ e) 
and c*d < 0. 

We claim, however, that every feasible direction d for HP^'-* satisfies c*d > 0. 
Indeed, otherwise the optimal objective value of HP).*' would be unbounded and 
hence there would exist x € relint(FeasJ.*') satisfying c*x < limsupc*a;j (using the 
assumption limsupc*a;j > — cx)). But x e Feas|j*.^ for sufficiently large j (because 
< pe (x) = limpe (x) for k ~ i, . . . ,Ti — 1), contradicting that Xj is optimal for 
HPW for aU j. 

Clearly, from the two preceding paragraphs, every limit point of {xj/UxjH} is 
contained in D := {d £ A^'g : Ad — and c*d = 0}. To show {a:^j/||a;j||} has a 
unique limit point it thus suffices to show D consists of a single ray. 

We claim DnA+je = 0. Otherwise there would exist d & D satisfying pi (d) > 
for k = i, . . . ,n — 1, which from continuity would imply d to be a feasible direction 
for HP).*' for all j > J (some J). But then Opt^*' would be an unbounded set for 

j > J, implying by Corollarv 14.2( 6') that Opt^*-* = Opt, and thus contradicting the 



assumed boundedness of Opt. Hence, it is indeed the case that D O A\^l^e = 0- 

It is clear now that D = {d £ 9A1j! e : Ad = and c*d = 0}. Thus, D is contained 
in a boundary face of A^'^g . Hence, D consists of a single ray or is a subset of A+ , 
by Corollarv l4.2r B). But if D were a subset of A+ then each d £ D would be a 
feasible direction for HP, which, with c*d ~ 0, would imply Opt to be unbounded, 
a contradiction to the assumed boundedness of Opt. Thus, D consists of a single 
ray, and so {a^j/Ha^jH} has a unique limit point d (and d satisfies d £ dA^'e, Ad — 0, 
c*d = 0). 

Lastly, we prove pe (d) < 0. 

As Xj £ dA^l'^f,., we have pe~ (xj) < (by p.SI) '). Thus, by homogeneity and 
continuity, pe \d) < 0. However, since d £ 9A\_*^e, if d satisfied pe (d) = 0, then 

we would have d £ A\^^e , and hence d £ A^ (by Theorem 14. 1 f A) ) - so d would be 
a feasible direction for HP, and hence Opt would be unbounded (using c*d = 0), a 
contradiction. Thus, pe '(d) < 0, completing the proof of the lemma. D 

Lemma 8.8. Assume Opt is a bounded set, e £ A_|__|_ and 1 < i < n — 2. Assume 
there exist sequences Cj £ Swath(i) and Xj £ Optg.' for which ej — > e, {xj} is 
unbounded and limsupc*Xj > — oo. 

(A) There exists a constant a < and an open neighborhood U of e such that 
for all e £ U O Swath(i) and x £ Optg , 

pt'\i) < a\\xr-^+\ 

(B) For each i £ M., there exists an open neighborhood V{i) of e such that for 
all e £ V{(.) n Swath(i) and x £ Opt^*\ it holds that \\x\\ > L 



Proof. Parts (A) and (B) of Lemma 18.71 together imply that if for one sequence 
{cj} C Swath(i) there exist Xj £ Opt).'.' for which {xj} is unbounded - where 
Cj — >■ e - then for every sequence {ej} C Swath(i) converging to e, and every choice 
of Xj £ Opt^*.\ we have liminf ||a;j|| — cx). Part (B) of the present lemma easily 
follows. 
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Towards proving part (A), let Cj and Xj be sequences as in the statement of 
the present lemma. Part (C) of Lemma 18.71 then shows Xj / 1 1 Xj 1 1 has a unique limit 
point d. 

We claim Sj/||a;j|| — > d for any sequences Cj and Xj £ Optg satisfying Bj — >■ 
e. Indeed, the intermingled sequences ei, ei, 62, 62, . . . and xi, xi, X2, a;2, . . . clearly 
satisfy the hypotheses of part (C) of Lemma 18. 71 and hence the normalized sequence 

a;i/||a;i||,Si/||Si|j,X2/|ja;2||,52/||S2||,... 
has a unique limit point, which, of course, is d, thus establishing the claim. 

Choosing, say, a — ^Pe (rf), part (A) of the present lemma is now a conse- 
quence of the continuity of the polynomial (e, x) i— )■ pg (x), and the fact that 
^ '"^ Pe {^) is homogeneous of degree n — i + 1 for all e. D 

Throughout the remainder of the section, 

• assume 1 < z < 71 — 2, 

• assume t 1— >■ e(i) (0 < i < T) is a maximal trajectory defined by the 
differential equation e(t) = x\L—e{t), starting at e(0) e Swath(i)\Core(i), 



• assume T < 00, and 



„(*) 



• to ease notation, let x{t) :— x,^y 



The third (and final) lemma for proving Proposition 18.41 shows that when the 
trajectory 1 1— >■ e(t) terminates at finite time T, the termination is not due to having 
reached the boundary of Peas. 

Lemma 8.9. {e(t) : < i < T} C A++ 



Proof. It suffices to show p{e{t)) > p(e(0))e^"* for all i, and hence suffices to show 
■^p{e{t)) > —np{e{t)), that is, suffices to show Dp{e{t))[e{t)] > —np{e{t)). Since 
e(t) = x(t) — e(t) and since Dp{e{t))[e{t)] — np{e{t)) (by (13.61) ). it suffices to show 
Dpieit))[x{t)]>0. 

However, by (|3.7p . for all e, Dp{e) is a positive multiple of Dpg~ {e). Since 
for all X, Dp" {e)[x] — pg (x) (because x >—>■ p" (x) is linear), we have 
Dp{e)[x] > if and only if x is in the cone A'^"^^^ But x{t) e ^+l(^t) ^ A^";'^^^', 
thus concluding the proof. D 

Proof of Proposition \8.4[ To rely on Lemmas 18.71 and 18. 8[ we need Opt to be a 
bounded set. Boundedncss of Opt, however, was established in Proposition 18.21 as 
a simple consequence of Swath(i) \ Core(z) being nonempty. 

Let £ be the set of limit points for the trajectory t 1— >■ e{t) as t — >■ T. By 
LemmaEH C C A++. Thus, if e G £, then either e € Core(i) (that is, Opt^'^ = Opt) 
or e e A++ \ Swath(i) (in which case Opt^*-' = 0). 

Assume first that C consists of a single point e. 

Then, by Lemma [8.7( A). each limit point of the path t 1— >■ x(t) lies in Opt^*-*. 
If the path is bounded, the set of limit points is nonempty, hence Opt^*-* 7^ 0, and 
thus, from above, e € Core(i). Clearly, then, if the path t h-> x{t) is bounded, case 
(A) of the proposition holds. 

On the other hand, if the path 1 1-^ x{t) is unbounded, then Lemma 15^ implies 
liminf(_>r |la:(i)|| = 00, and implies there exists < t < T and a < satisfying 

Pem\^it)) < c^MW'"^^ for ah i < t < T (8.4) 

(here we make use of the fact that t h^ c*x{t) is strictly increasing fProposition l8.2l) . 
so that the hypothesis limsupc*Xj > — cx) of the lemma is clearly fulfilled). Hence, 
from the compactness of the closed interval [0,t\ and the fact that for all t, both 
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x{t) 7^ (because 6^0) and p|;!~^ {x{t)) < (by p. 41) ). a perhaps larger (but still 
negative) value of a satisfies 

P%)\xit)) < a\\xit)r-'+^ for alio <i<r. (8.5) 

Hence, in all, if the path t i— )■ x{t) is unbounded, then case (B) of the proposition 
holds, concluding consideration of the case that £ consists of a single point. 

For the remainder of the proof assume C contains more than one point. We 
show case (B) of the proposition holds. We claim that for this it suffices to show 
C is compact, and that each e G £ satisfies the hypotheses of Lemma 15^ Indeed, 
compactness and Lemma [HH A) imply C can be covered by finitely many open sets 
Uj for which there exist aj < with the property that for all e G UjD Swath(i) and 

X G Optg , it holds pg '{x) < aj\\x\\"^~^~^^ . Consequently, letting a :— maxjOfj, 
there exists < f < T for which (|8.4p holds. Then, as in the preceding paragraph, 
for a possibly larger (but still negative) value of a, (jB.Sp holds. To conclude proving 
the claim, it remains only to show liminit^T 11^(^^)11 = c». This, however, is easily 
accomplished by covering £, for each ^ G M, by finitely many open sets V{£)j as 
appear in Lemma l8.8f B). 

Thus, to complete the proof of Proposition 18.41 it remains only to show £ is 
compact, and that each e G £ satisfies the hypotheses of Lemma [8.81 

That C is compact is trivial - indeed, the trajectory t i— > e(<) is bounded, by 
Proposition l8.2l 

Since C consists of more than one point, for each e G £ and each open neigh- 
borhood W of e, the Euclidean arc length of {e(i) : < t < T} n W^ is infinite. 
Thus, since T < oo, for each e G £ there exists an sequence ti < t2 < ■ ■ ■ satisfying 
e{tj) — > e and ||e(tj-)|| — >■ cx). But e(tj) = x(tj) — e{tj), so ||a;(<j)|| — ?> oo. Since, 
additionally, limsupc*a;(ij) > — oo (because t i— >■ c*x{t) is strictly increasing), e 
thus clearly satisfies the hypotheses of Lemma 18.81 D 



8.2.2. Proving the second of the two propositions. In proving Proposition l8.5[ 
continue to 

• assume I < i < n — 2, 



assume t i— > e{t) {0 < t < T) is a, maximal trajectory defined by the 
differential equation 
• assume T < oo, and 



differential equation e(t) — xlL ~e(i), starting at e(0) G Swath(i)\Core(i), 



• to ease notation, let x{t) :— x^^y 

Also, for every e, define Qe := pi /pi ■ 

The proof of Proposition 18.51 depends on two lemmas. For understanding the 
first lemma, recall the identity given as (j8.1|) . that is. 



D,{D^p^^\x)) ^ kD'+^p^^-^\x) . (8.6) 

Lemma 8.10. For any e and x satisfying pi {x) — Q^pi {x), we have 

D,{Dq,{x))[x~e]=Dqe{x) 

i(n — i) „ ti T\ , , 
+ ,\,, ' Dpi'-^\x) 
pi'^'\x) 

_ i{n-i + l)pt'\x) ,^^,^ 

( (*+i)^ ^^2 ^^ ^ ' ■ 
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Proof. For an arbitrary vector Ae and for any x at which Qg is defined (i.e., any x 
satisfying pe {x) ^ 0), use of (|8.6|) gives 

D,{Dq,ix))[Ae] = D, ( i^I?pW(^) _ pij^ ^p(.+i)(^)A [^^j 

(pi'+''(a;))2 ^e V yi J 

Substitute Ae = x— e, then use obvious identities to get rid of "[e]" (e.g., Dpe (x) [e] — 
Pe (x), D'^pe {x)[e] = Dpe'{x)), and use homogeneity to get rid of "[a;]" 
- specifically, use (13.61) . Finally, substitute pe (x) = 0, and -jiJh — Dpe (x) — 

Dqe{x) (because pe (x) — 0), thereby concluding the proof. D 

Lemma 8.11. For all < t < T, there exists y{t)* satisfying 

°''-')'-"»'^"'l- (..(.W-'!w))^ °' 

Proof. CoroUarv 17.41 shows x{t) is optimal for the convex optimization problem 

min^; - In c* (e(i) - a;) - qe{t) {x) 
s.t. Ax = b 

and hence there exists 'w{t)* satisfying the first-order condition 

^^;^^r^^'^*-Di<^Mt))--^i^r^- (8.7) 

Since 1 1— )■ e{t) and i i— > xy^.) are analytic, so is i i— > 'U'*(i) (using that A is surjective, 
by assumption). Differentiating in t gives 

^l^f^l^c* +i52g,(,)(x(t))[i(t)] +i?,(,)(i^g,(,)(x(t)))[e(t)] = -u;(i)M . 
To complete the proof, substitute 

e{t) = x{t) - e{t) and ^ _ c* ^ Dqe(t){x{t)) + ^;(t)M (by (HID) , 

and then use Lemma 15. 101 to substitute for De(t)(Z?qe(()(x(i)))[x(i) — e(i)]. D 

Proof of Provosition \8.5l To temper notation, for arbitrary t satisfying < t < 
T, let 

e := e{t), x := a;(t), e := e(<) and i := x{t). 
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Observe that 



dt '" / (j+l), .,^^^"-*+l 



(- - ^ In (-pl;7/^(-(t))) -{n-^ + l)i \np%'\x{t)) 



Using Lcmma[8H]to substitute for ,,"7;' Dpi' ^'(a;) - f~i'+\ i:)pi'+^^(g;) gives 



p:,'""'(x) ' '■ ' pi^ '{x) 



n— z 



dt "^ ( (^+l), .,xA"-'+l 



ipt'Kx) \ " ' ^ \c*{e-x) 

. (n - .)^^.;;MM .(._., .,^4^ (using ,*,)..,) . 0) 

Pe [x) pi '{x) 



zpi'-'^x) Vc*(e-2;); pi'-')(x) pi''+'^(a;) 

where the inequality is due to the combination of pe {x)/pe {x) being nega- 
tive (according to p.4p and (|4.2p ) and D'^qeix) being negative semidcfinitc (Theo- 
rem E^]). 

To complete the proof, simply substitute the following expressions which result 
from use of e = x — e, p.6p and (|8.ip : 



(.-l)((n-z + 2)p('-2)(x)-p('"i)(x)) , 



Z?e(pi'+'H2;))[e] = (i + l)Dp^:H^)[x - e_ 
= (z + l)(0-pi'+i)(x) 
substitute, also. 



ex 



c*(e — a;) 



= Dqe{x)[x\ (first-order condition arising from Corollarv l7.4p 

Dpi' (x)\x\ , . (i\, s „N 

- u+\ \ (using pW(x)=0) 

Pe Ha;) 

_ ^e(pi'^(a:))[e] /because ^^b^'^ frc , ^ W 0^ 

- (i+i), , loecause ^Pg(j^(_a;e(f)j -Uj 

Pe (x) 

Dpe (a;) [a; — e] 
^"' Pi^+^'(x) 

(n-?: + l)pi'"^^(x)-0 



D 
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9. Proof of Part II of the Main Theorem 

Recall the optimization problem dual to HP: 

supy. ,. y*b ^ 

s.t. y*A + s* =c* \ HP* , 

s* e a; J 

where A* is the cone dual to A^_. Recall that just as a matter of definitions, the 
optimal value of HP* satisfies val* < val ("weak duality"), where val is the optimal 
value of HP. 

Recall that a pair {y* , s*) satisfying the constraints is said to be "strictly" feasible 
if s* e int(A* ) (interior). 

For e G Swath(i) \ Core(i), define 



i"' := ^-Ajy^Dp^^ (^) where x = x« 



We now restate and prove IPart III of the Main Theorem. 

Theorem 9.1. Assume 1 < i < n — 2 and let {e{t) : < i < T} be a maximal 
trajectory for the dynamics e{t) — xlL — e{t) starting at e(0) € Swath(i) \ Core(i). 

Then y* A + s A = c* has a unique solution y* — y f\\, and the pair (y Z-,, s L-,) is 
strictly feasible for HP*. Moreover, 

y i\sb — c*x Ls > val 

ye(t) e(t) ^^rp 

(in fact, increases to val strictly monotonically) and the path t i— > (2/e/f)j Sefji) is 
bounded. 

Proof. Assume e € Swath(i) \ Core(i). By Corollarv 17.41 Xe is optimal for the 
convex optimization problem 

mm^ - In c (e - x) '+i\ ,\ 

Pc {^) 

s.t. Ax — b , 
and hence there exists y = ye satisfying the (rearranged) first-order condition 

A*y + sf^c, 
where, letting q^ := pe /pe and x^ = xj" , 

.« := ic*ie-Xe))Dqe{xe) = ^!|-^i?pW(xe), 

Pe [Xe) 

(using Pe (xe) = 0). Thus, to show the pair (j/e , si) is strictly feasible for HP*, it 
sufiices to show Se G int(A*), that is, assuming z € A+, z 7^ 0, it suffices to show 

Dqe{Xe)[z] > 0. 

Corollary 14.21 shows (9A\*^e) \ A+ contains no line segments of positive length 
other than those lying in rays {sx : s > 0}. Thus, the line segment connecting Xe 
to z lies entirely within A^'je with the exception of the point Xe and possibly the 
point z. Hence, z{s) := (1 — s)z + sxe satisfies qe{z{s)) > for < s < 1. Fix s 
strictly between and 1. 

Concavity of qe (according to Theorem 17. 2p and qeixe) — imply qe{z{s)) < 
Dqe{xe)[z{s) — Xe], whereas qe{xe) = and homogeneity of ge give Dqe{xe)[xe\ = 0. 
Thus, 

< qe{z{s)) < DqeiXe)[z{s) - Xe] = (1 - s)Dqe{Xe)[z] , 
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completing the proof that (j/e , Se ) is strictly feasible for HP*. Additionally ob- 
serving 

— C Xq 5g Xq 

= C*Xe - (C*(e - Xe)) Dqe{Xe)[Xe] 

* 

— C Xq , 

we thus see for a maximal trajectory {e{t) : < t < T}, each pair (2/e(f), s^/M is 
strictly feasible for HP*, and 

(*)/,_ * (i) 

— >■ val strictly monotonically (by Proposition 18. 21) . 

It only remains to show the path 1 1-> (ygVj-,, 3^) is bounded, for which it sufhces 

to show 1 1— >■ slL is bounded (as A is surjective, by assumption). In turn, because 

e(0) S A_|_+ and slL S A* , it sufhces to show the value Sgfj')e(O) is bounded from 
above independent of t. However, 

4)6(0) =c*e(0)-y« MO) 

->■ c*e(0) - val , 
concluding the proof. D 

10. Proof of Theorem 6 

We have now finished our analysis of the trajectories t i-> e{t) (and paths t i-> 
xlL) arising from the differential equation e(i) = x\L — e{t) on the set Swath(i) \ 
Core(i). However, in order to have a relatively complete picture of all of Swath(i), 
results regarding the structure of Core(i) are needed. That is the purpose of this 
section. 

Specifically, just as the dynamics e(i) = xV^. — e{t) on Swath(i) \ Core(i) leads 
to optimality, it would be nice as a matter of formalism to know that if e G Core(i) 
and x e Opt, then moving e towards x results in a point e + t{x — e) (0 < t < 1) 
also in Core(i), thus retaining optimality. 

Theorem 10.1. Assume 1 < i < n — 2. If e £ Core(i) and x E Opt then 

{e + t{x - e) e A++ : t e M} C Core(i) . 

Extending the theorem to include i = is trivial, because Core(O) is precisely the 
interior of Feas. It also is easy to extend the theorem to i = n — 1, simply because 
Core(n — 1) = 00 We state the theorem only for l<i<n — 2to avoid having to 
be mindful of the special cases i = 0,n — 1 during the course of the proof. 

By bootstrapping the theorem, the following elementary corollary provides some 
additional insight into the structure of Core(i). The corollary is a restatement of 
Theorem |6l 



To see Core(n — 1) = 0, recall that (SA^^ ) n A-|. is precisely the lineality space of A4. 
(by Theorem 14. ir O). Thus, since A^ is regular (by assumption) and since the origin is infeasible 
(because b ^ 0), the boundary of Feas cannot intersect the boundary of FeaSg ; in particular, 

it cannot happen that Opt intersects OptJ;" 
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Corollary 10.2. //e G Core(i) and A is the smallest affine space containing both 
e and Opt, then 

A n A++ C Core(i) . 

Proof. If Opt contains only a single point, then the corollary is nothing more than 
a restatement of the theorem. Thus, assume Opt contains more than a single point, 
and fix a; G rclint(Opt). Then 

A = {x + a{e- x) + I3{x- x) : aeR, 13 >0 and x G Opt} . 

Assume y £ Ad A++, and fix a G M, /3 > and x G Opt satisfying 

y ~ X + a{e — x) + l3 {x — x) . 

To complete the proof, we show y G Core(i). 

Since y and e lie in the interior of Feas, we have c*y, c*e > val (— c*x, c*x), from 
which follows a > 0. Thus, the point 

is well-defined, is a convex combination of e and x, and lies in A++ (because e G A+^., 

a; G A+ and a > 0). As e G Core(i) and x G Opt, Theorem 110.11 implies z G Core(i). 

However, y — z + t{x — z) for t = a + j3 — \. Hence, applying Theorem 110.11 

again, now with z (resp., x) in place of e (resp., x), we find y G Core(i). D 

Before turning to the proof of Theorem 110.11 we recall the conjecture made in 

m 

Conjecture. Core(i) is convex. 

It is highly unclear whether resolving the conjecture positively could have any 
relevance to algorithms. Nonetheless, a positive resolution would be interesting 
from a structural perspective. 

Now we begin the process of proving Theorem 110.11 For motivation, we first 
consider a reasonably-general case for which the proof is rather straightforward, 
and afterwards move to the proof for the truly general case. That the proof is 
straightforward in one setting but not in full generality is reminiscent of the proof 
of Part I of the Main Theorem, where the case of "T = oo" was vastly easier than 
the case "T < oo" (i.e., ijS.ll was a breeze compared to ^8.2p . 

The straightforward case relies on the following lemma. 

Lemma 10.3. If Q <i <n~2 and x G dKl]^, then 

Dp'i\x)=Q <^ x(^dK^l%^K 

Proof. This is Lemma 7 in fT3]. □ 

Here is the straightforward case: 

Theorem 10.4. Assume 1 < i < n — 2 and e G Core(i). If x £ Opt n A++.e then 
{e + t{x - e) G A++ : t G M} C Core(i) . 

Proof. Recall Corollarv l7.4[ which shows that for every e G A+_|_ and < i < n — 2, 

the points lying in Optg \ dA^ g are precisely the optimal solutions to the linearly 
constrained, convex optimization problem 

'^'"'-^A'ix:! ~ ^"^ "* (^ - ^) - #^ 

s.t. Ax — h , 
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that is, for every e S A^.+ , letting q^ ' := p\ 7P! 

^.(g-:.) c* - ^9g'^ (a;) ^y*A for some y* 
Ax = b 



(10.1) 



Now assume x G Optn A+^^e , where e € Core(i) and < i < n — 2. Then, since 
A+ n 9A^^g (0 < j < n - 1) is independent of e S A+_|_ (by Theorem |41T B)). we 
have for all e G A++ that 

X e K^:t^^ , (10.2) 

p(,j)(a;)=0 forallO<j<i, (10.3) 

and, using Lemma 110.31 

Dpi^\x)=0 for alio <j < i- 1 . (10.4) 

Moreover, by (jlO.ip . there exists y* satisfying 

-,^c^-D<^{x)^y*A, 

that is, as pi {x) ~ 0, there exists y* satisfying 

Fix t for which e + t{x — e) e A++ (thus, — oo <t<\). To prove the theorem, it 
suffices to show x G OT>\l.t{x-e)^ because then. Opt n Opt^*^^, ^^^ ^ 0, and hence. 
Opt = Opt^'|^(^__^^ (by CoroUaryimB)). Thus, in light of ([Tin)) , we see from 
(|10.ip that to prove the theorem, it suffices to show 

7T — Trhn V c* — Dq^'. ., Ax) — w* A for some w* , 

that is, as P^j^tix-e)^^) ~ ^ (^^ (110. 3^ *1. it suffices to show 

,, ,, \, . c* - -oT^ Dp^^i,, ^{x) = w*A for some w* . (10.6) 

However, 

T^V%(^.,p) = D'^^p{x)\ {\ - t)e + te, . . . , (1 - t)e + fa ] 

i times 

= E Q) (1 - i)-''t*--'^D'+V(a;)[e^_^, x__^] 

■^~ j times i — 'j times 



^ r. j (1 - tyf-w'+^-^pi^'^ {x)[x,...,x] 

t — J times 

±Qa-m-^^^^^^Dpi^\x) (by 

{i-tyopi^Hx) (bydina). 
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and, similarly, 








^e)(-) 


U\ 3 r ^ {n-^-l)\ 
= (l-t)^+Vi'+i)(x) (byUiLal). 



pw(x) 



Hence, w* = -^ y* satisfies ()10.6p (where y* is as in (jlO.Sp '). completing the proof. 

D 

In the course of the proof for the "truly general" case, there is a crucial outside 
result to which we refer, and thus before formally beginning the proof, we explain 
the result. 

For e e A++ and an arbitrary point x, let Me{x) denote the number of non- 
negative roots of i i~-> p{x + te). The result is that Me(x) is independent of e € A++, 
as was stated in [9 (as Theorem 2.12). (In ^3j, the fact was (essentially) established 
only for x G A+.) 

Perhaps worth recording is that the independence easily follows from a most 
useful tool in the hyperbolic polynomial literature: 

Helton- Vinnikov Theorem. Assume e e A_|_^. For any points x and z, there 
exist n X n symmetric matrices X and Z such that 

(r, s, t) i-> pirx + sz + te) — p[e) det(rA' + sZ + tl) . 

This formulation of the theorem comes from |11| , where it was obtained by straight- 
forward homogenization of the original result in |10| . (The initial importance of 
the homogeneous version was that it affirmatively settled the "Lax Conjecture" ~ 
see [TT] for discussion.) (See [3] for negative results on possibilities of extensions to 
more than three variables r, s, t.) 

Corollary 10.6. For e G A++ and arbitrary x, let Me(x) denote the number of 
non-negative roots for the polynomial t i— >■ p{x -\- te). The value Me{x) is the same 
for all e G A_|__|_. 

Proof. First observe that because all roots of univariate polynomials vary continu- 
ously in the coefficients so long as the leading coefficient does not vanish, it suffices 
to show multg (x) is independent of e G A_|__|_ , where multg (x) is the multiplicity of 
as a root of i i->- p{x + te). However, for e, z G A++, using the Helton- Vinnikov 
Theorem, 

muhe(x) =n- rank(X) = n - rank(Z-i/2^^-i/2>) ^ niuh3(a;) , 
and thus the corollary is established. D 

Proof of Theorem \10.1[ Fix e G Core(i), where 1 < i < n — 2. Let x denote a 
point in Opt (= Opt^'^). 

Let e{t) := e-\- t{x — e), and let / denote the open interval consisting of alH G K 
for which e{t) G A++ - thus, / C (— oo, 1). 

Since A+ n dA]^'g is independent of e G A_|_+ (by Corollary 14. 2r A)). and since 
Opt^'^ = Opt, we have 

Opt C ^Af^^^^ for aU t G / . (10.7) 

Let V := {d : Ad — and c*d — 0}, and let x +T> he the set consisting of all 
points X + d where d € V. 

Since relint(Feas^*(^jP ^ {x + {d : Ad = 0}) n A^'| ^^^^ and x G Opt C Feas^'^^^^, we 
have 

xGOptg) ^ (x + p)nA«^^(,)=0. 
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On the other hand, x is in Opt^^Z-,, as well as in Opt, if and only if e(t) e Core(z) 
(by Corollary 14.2^ 6) ) . Consequently, 

e(i) e Core(^) ^ {x + V)C^ h-f^^^^t^ = ■ 

Thus, our goal - proving e{t) £ Core(j) for all i S / - will be accomplished if we 
prove (x + V) n A ,^. = for all t <E I. Hence, fixing arbitrary d £ V, letting 
ray((i) := {sd : s > 0} and 

I{d) :^{tel: (x + ray(d)) n A« ^^^^^ = 0} , 




our goal is to show I{d) = I. 

Recall, however, the characterization p.2p now applied to A | ,^-. : 

A« _^(,) := {x : p(||) (x) > for aU j = *, . . . , n - 1} . (10.8) 

Clearly, then, I\I{d) is an open subset of the open interval /. Hence, since I{d) ^ 
(indeed, € I{d)), to complete the proof of the theorem, it suffices to show that 
I{d) is open. As e is only assumed to be an element of Core(i) - in particular, 
e could be replaced with any e{t) that happens to be in Core(i) - our goal has 
become: 

Show I{d) contains an open interval including 0. (10.9) 

For non- negative integers j and fc, define 

ajkit):=D''p['^^^{x)[d,...,d] 

k times 



if ajk{t) = for all k ; 

otherwise, where k{j,t) := min{fc : ajk{t) =/= 0} . 

Note that if (3j{t) = 0, then the polynomial s !-> plL{x + sd) is identically zero, 
whereas if /3j{t) ^ 0, the polynomial evaluated at small, positive s has the same sign 
as Pj{t). Thus, since x lies in the convex set A^^^.^s, we have by the characterization 
(flail that 

{x + ray(d)) n A^^j^^^^^ ^0 ^ /3, (t) > for all j = z, . . . , n - 1 . 

Our goal (110. 9p has now been reduced to: 

Show 3e > such that |i| < e ^ (ij{t) < for some j E {i, . . . ,n - 1} . (10.10) 

For this we consider two cases. 

First, assume /3j(0) > for all j = i, . . . , n — 1. Then x + sd E Feas^*-* for all 
sufficiently small, positive s. Thus, since x G OptJ.*^ = Opt and c*d — 0, we have 
x + sd £ Opt for all sufficiently small, positive s. Hence, by (jl0.7p , for each i g /, 
the univariate polynomial s i— ^ Peit) (-^ + *^) ^^^ value zero on an open interval, and 
thus is identically zero, so I3i{t) = for all t £ I, (more than) accomplishing (|10.10p 
for the first case. 

Now consider the remaining case, that is, assume /3j(0) < for some j G 
{«,...,«— 1}; fix such a j. To accomphsh ()10.10p , of course it suffices to show for 
this fixed value of j that /3j{t) < for all t in an open interval containing 0. In 
turn, by definition of fij{t), it suffices to: 

Show 3e > such that \t\ < e ^ D^^''°^p^^f^^{x)\ d,...,d]<0 (10.11) 

k(j, 0) times 
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and 



show (t e /) A (fc < /i:(j, 0)) => L>Vl{i)(a;)K...,rf] =0. (10.12) 

k times 

The existence of e as in (|10.1ip is simply a matter of continuity and /?j(0) < 0. 
For accompUshing ()10.12p , observe 

i:»'=pgj(x) = D^+^p{x)[{l - t)e + to, . ■ . , (1 - t)e + tx] 



j times 



i^O 



^ (;)(i-t)v-^D'=+^--M^)(.)[x__3] 

^ — j times 



Consequently, (jl0.12p is immediately accomplished by the following proposition, 
thus concluding the proof of the theorem (except for proving the proposition) . D 

Proposition 10.7. Assume e e A++, x G A+ and let d he a vector. Assume non- 
negative integers j and k satisfy D^pJ [x)[d, . . . ,d] ^ 0; let k{j) be the smallest 

k times 

such k for j and assume k{j) > 0. 
Then 

D^p^^\x)[d, . . . , d] = for alli = 0,...J andk = 0,..., k{j) - 1 . 

k times 

The proof of the proposition makes use of the following lemma. 
For e G A++ and arbitrary z, recall that Me{z) denotes the number of non- 
negative roots of t iH- p{z + te). 

Lemma 10.8. Assume e, a; G A_^^, let d ^ Q, and let Si < S2 < • • ■ < sj. denote the 
positive roots (including multiplicities) of s i-^ p(x + sd). Then, for every s > 0, 

Me{x + sd) = if{i : Si<s} . 

Proof. Because Mf,{x + sd) is independent of e G A+^ (by Corollary [TU^, in prov- 
ing the lemma we may assume e = x, in which case we need only consider the 
hyperbolic polynomial obtained by restricting p to the subspace spanned by x and 
d. However, every hyperbolic polynomial in two variables of degree n is of the form 
(2/1,2/2) '^ TYj=i '^Jy foi' some vectors aj G M."^ (this easily follows from two facts: 
(i) every complex homogeneous polynomial in two variables of degree n is of the 
form (2/1,2/2) '-^ n?=i(%',i2/i + 0^,12/2) for some Qj G C^; (ii) if a G C^ is not a 
(complex) multiple of a real vector, then {(2/1, 2/2) £ R^ : ai2/i + 0^22/2 — 0} contains 
only the origin). 

Thus, we need only consider hyperbolic polynomials ^(2/1,2/2) = TVi=i'^Jy ~ 
where aj G M^ - and x = (xi,a;2) which for all j satisfies ajx ^ 0. For {di,d2) 7^ 
(0, 0), our goal is to show for s > that the number of roots < s < s (counting 
multiplicities) for the univariate polynomial 

n 

s^YlaJ{x + sd) (10.13) 

is the same as the number of roots t > (counting multiplicities) for 

n 

t^Y[aJ{x + sd + tx) . (10.14) 

i=i 
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Of course, however, s > is a root of (|10.13p if and only if s = s/(l + i) for some 
root £ > of ()10.14|) . The lemma follows. D 

Proof of Proposition \ 1 0T7\ Assume e G A^^. For arbitrary z and < £ < n — 1, 
let Me '{z) denote the number of non-negative roots (counting multiplicities) for 
t<-^pi'{x + te). Additionally, for a vector d and value s > 0, let Ni '{x,d,s) denote 
the number of roots (counting multiplicities) in the closed interval [— s, s] for the 
univariate polynomial s i— >■ pi (x + sd) ; if the univariate polynomial is identically 
zero, let Nj; {x,d,s) = cx). 

lix e A++ (hence x G A^'^j^e for aU < £ < n - 1), d ^ and s > 0, Lemma [TOSl 
can be applied with pe in place of p, and applied with — d as well as with d, yielding 

Ni'^'>{x,d,s) = Mi^Hx + sd)+Mi'^'>{x-sd) . (10.15) 

On the other hand, for any z, the interlacing of the roots oi t >—>■ pe {z + te) and 
its derivative t H> pe {z + te) gives Me{z) > Me^'{z), and thus, 

£<j => Mi'^\z)>M'^J'>{z) . (10.16) 

From PO.ISP and (|10.16p follows 

{x e A++) A {£ < j) ^ Ni'^'>{x,d,s)>Ni^'>{x,d,s) . (10.17) 

Assume, now, that x, d, j and k(j) satisfy the hypothesis of the proposition. 
Observe that definitions readily give k{j) = Ng (x,(i, 0). Moreover, proving the 
proposition amounts precisely to showing 

N'-f\x,d,0)>N'-J\x,d,0) for£ = 0, ...,j . (10.18) 

For e > 0, let x{e) := x + ee, a point in A++. Each polynomial s i-^ pi {x{e) + 
sd) (0 < £ < n — 1) is not identically zero and has only real roots (indeed, by 
homogeneity, the roots are the reciprocals of the non-zero roots for t i-> pi (d + 
te(e))). Thus, since 

bounded roots of a univariate polynomial "] 

vary continuously in the coefficients > (10.19) 

(so long as the polynomial does not become identically zero) J 

we have for < £ < j that either Ni (x,d, 0) = oo (i.e., s i-^ pi [x + sd) = 0), or 
Ni^\x,d,0) = lim Mim ('NW(x(e),d, s) 



im(N(^")(a:(e),d,s))] {hy mM) 



> lim lim 

> lim (Ni^) {x, d, s/2)) (again using ^UJ^) 

= N(^)(x,d,0), 
thereby establishing (|10.18|) and hence completing the proof. D 
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